OpenPilot

Jason,

Just no watching Harry Potter okay?

I have been considering this for my 2016 e-golf SE that does not seem to have a camera. I got two of these J533 Harnesses from aliexpress even though I ordered one:
Nice to see consideration of supporting vision only for cars without ACC.

There's somebody (Saber422 on Discord I think) having fully complete harnesses built out of China, but the QA on the first round was atrocious. Hearing the next round will hopefully be better. I've actually got comma themselves talked into making official J533 harnesses, but they're not quite ready for sale yet.

Longitiudinal control is my next major project, now that comma are back to investing more in longitudinal training for their model. I have an old test-mockup version that worked from about a year ago, but the vision model wasn't yet a clear win over stock ACC, so I never finished some things like transition in-and-out of standstill.

Got back to working on the MQB specific longitudinal control code.

Coupled with the latest end-to-end vision model, ignoring the stock ACC radar entirely, cool stuff becomes possible!

jyoung8607 said:

Coupled with the latest end-to-end vision model, ignoring the stock ACC radar entirely, cool stuff becomes possible!

Click to expand...

Neat, but that particular situation, specifically the self-start at the intersection strikes me as needing more than just a forward-looking camera with a limited field of view because it will surely not see the occasional idiot who decided to run the yellow that just turned red on the perpendicular road...?

-Uwe-

Uwe said:

Neat, but that particular situation, specifically the self-start at the intersection strikes me as needing more than just a forward-looking camera with a limited field of view because it will surely not see the occasional idiot who decided to run the yellow that just turned red on the perpendicular road...?

Click to expand...

A valid question. The newer comma three has three cameras: forward narrow angle, forward wide angle, and reverse wide angle. The field of view for the forward and reverse wide cams actually overlap very slightly, so it sees all around. It can see cross traffic better than I can. Stoplines are still at "cool party trick" level of maturity, something you have to go way far out of your way to deliberately enable, but the limitations are training and model development, not FoV.

Here are the raw images from each, at that moment in time. They don't look as good as they should because it's night and HDR is disabled, bad pixel mapping is off, and my windshield is dirty AF.

Forward narrow:



Forward wide:



Reverse wide:

jyoung8607 said:

The newer comma three has three cameras: forward narrow angle, forward wide angle, and reverse wide angle.

Click to expand...

Ah. I suppose I incorrectly assumed that it was working only from the data displayed on its own screen that I could see in the video, which appears to be the "forward narrow"..?

-Uwe-

Uwe said:

Ah. I suppose I incorrectly assumed that it was working only from the data displayed on its own screen that I could see in the video, which appears to be the "forward narrow"..?

Click to expand...

Yeah, but the model drives on the view from both front cameras now. The intent is to actually drive with all three, to supplement the blindspot radars with a peek out the side windows, but that isn't happening yet.

A better FoV demo from the Tail of the Dragon -- front narrow vs front wide cam view of the same 60 seconds of driving. It sees around hairpins better than the driver can.

jyoung8607 said:

A better FoV demo from the Tail of the Dragon -- front narrow vs front wide cam view of the same 60 seconds of driving. It sees around hairpins better than the driver can.

Click to expand...

How dependent is it on those nicely painted, highly visible lines?

You're probably too young (pun intended) to remember this 1976 arcade game:


But I bet OpenPilot could get a perfect score!

-Uwe-

Uwe said:

How dependent is it on those nicely painted, highly visible lines?

You're probably too young (pun intended) to remember this 1976 arcade game:

But I bet OpenPilot could get a perfect score!

-Uwe-

Click to expand...

I remember coding a Night Driver-esque game in BASIC during a summer computer class my school ran back in the early 80s. I thought it was cool until I tried to play it.

This is a talk I gave at comma_con back at the end of July, the high-quality replay just went live on their channel. I explain how openpilot integrates into a car and how it gains support for new cars. Many of my examples are drawn from a Volkswagen world, but it's meant to apply to most modern cars.

Contains one stolen photo from the forum proprietor, meant to go with a dedicated slide about use of diagnostic tools like VCDS in CAN reverse engineering that had to be cut for time. Sorry.