Sensor never starts streaming data 70% of the time on X1 Carbon Gen 11 #187
Comments
|
+1 on this, same symptoms x1 carbon Gen 10 i7-1270P Let me know if you need me to add any other info about my hardware. |
|
@lukemarsden so I was doing some IPU6 work today and this made me think back to this bug. The ov2740 driver has 2 sets of register settings one for Lenovo devices and one for other devices. Which set of register settings to use is decided by making a special ACPI call which gets a camera-sensor-module id string. I wonder if the X1 Gen 10 / Gen 11 maybe has a different sensor-module (different sensor module ID) causing the ov2740 to use the non Lenovo register settings which in turn then causes this problem. Here is an updated akmod with a patch to try this: After downloading run the following commands to install it: After this wait for the new kmod to be build and then run, e.g.: rpm -qa | grep kmod-intel-ipu6 This should show the kmod-intel-ipu6 release field for your currently running kernel is now After this reboot and do: dmesg | grep "Sensor module id" This should now show your sensor module id. If your sensor-module id is |
|
Thanks for thinking of this bug!
Unfortunately:
[ 6.854496] ov2740 i2c-INT3474:01: Sensor module id: 'CJFLE23'
Any other thoughts about what might be causing this? Happy to run commands
and report back :-)
Cheers,
Luke
…On Tue, Nov 7, 2023 at 11:59 AM Hans de Goede ***@***.***> wrote:
@lukemarsden <https://github.com/lukemarsden> so I was doing some IPU6
work today and this made me think back to this bug. The ov2740 driver has 2
sets of register settings one for Lenovo devices and one for other devices.
Which set of register settings to use is decided by making a special ACPI
call which gets a camera-sensor-module id string. I wonder if the X1 Gen 10
/ Gen 11 maybe has a different sensor-module (different sensor module ID)
causing the ov2740 to use the non Lenovo register settings which in turn
then causes this problem.
Here is an updated akmod with a patch to try this:
akmod-intel-ipu6-0.0-11.20230622git8e41080.fc38.x86_64.rpm.gz
<https://github.com/intel/ipu6-drivers/files/13279525/akmod-intel-ipu6-0.0-11.20230622git8e41080.fc38.x86_64.rpm.gz>
After downloading run the following commands to install it:
gzip -d akmod-intel-ipu6-0.0-11.20230622git8e41080.fc38.x86_64.rpm.gz
sudo rpm -Uvh akmod-intel-ipu6-0.0-11.20230622git8e41080.fc38.x86_64.rpm
After this wait for the new kmod to be build and then run, e.g.:
rpm -qa | grep kmod-intel-ipu6
This should show the kmod-intel-ipu6 release field for your currently
running kernel is now 11.20230622git8e41080 this will be part of the full
string for the package.
After this reboot and do:
dmesg | grep "Sensor module id"
This should now show your sensor module id. If your sensor-module id is
CJFLE23 then this patched akmod pkg will not make a difference. But if it
is a different id, then this patch will hopefully help to make the camera
work better.
—
Reply to this email directly, view it on GitHub
<#187 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACATUVRJYRNSO273NRBYILYDIPAPAVCNFSM6AAAAAA5TDSMNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJYGM3DGNRUGA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
lukemarsden.net
***@***.***
@lmarsden <https://twitter.com/lmarsden>
|
Sorry, but I don't have any other ideas how to fix this. |
|
@lukemarsden Could you try adding "options intel-ipu6-isys dyndbg=+p" to /etc/modprobe.d/intel-ipu6.conf then restart to check dmesg log again? We can check if IPU received any frames.\ By the way, the settings used for module CJFLE23 should be 180M link frequency and I fixed that on master branch. It might helps but need a try. |
|
Thank you @hao-yao Here's a video of what happens when I set that option, reboot, then run 2023-11-18_12-11-57.mp4I've also uploaded it to YouTube here: https://www.youtube.com/watch?v=OQpBTUpM4sk |
@jwrdegoede is this change something you might be able to conveniently package for me to try please? 😄 |
@lukemarsden This shows that IPU didn't receive CSI-2 packages. My experience is that the issue should between camera sensor module and IPU CSI-2 port, either register settings or link frequency wrong, or they don't match. |
Here is an updated akmod: https://fedorapeople.org/~jwrdegoede/akmod-intel-ipu6-0.0-11.20231115git067270f.fc39.x86_64.rpm Download it and then run Hopefully this will resolve the X1 Carbon gen 10 / gen 11 issues. |
|
Thanks! Unfortunately I'm now getting this: Is it because I'm still on Fedora 38? I noticed the rpm was fc39. dmesg: https://gist.github.com/lukemarsden/7fa4f4e720c01a1452a604fc0a00b08c Linux fixed 6.5.10-200.fc38.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Nov 2 19:59:55 UTC 2023 x86_64 GNU/Linux |
|
Oh, after rebooting twice, it's now working! At least, it's working MORE reliably than it was before, I think, but still not 100%... |
|
Running 1 - worked straight away So yeah we're still at about 30% - I'd say this change unfortunately hasn't made a difference. Any other logs etc I can share? |
|
Some more dmesg after all those tries: https://gist.github.com/lukemarsden/f8c21affa6f0a0ffd5e3d6550208583f |
|
That is unfortunate, can you run: And check the printed kmod version ends with "0.0-11.20231115git067270f.fc39" ? This checks that you are indeed running the new module. |
|
Yup, |
|
I tested upgrading my X1 Carbon Gen 11 to Fedora 39 and even upgraded to 6.6.2-201.fc39.x86_64 kernel and still getting the same symptoms. Strangely enough, my X1 Yoga Gen 8 running Fedora 38 seems to have regressed to the same behavior described in this issue - I had the kernel pinned to 6.5.5 but the userland was upgrading. I wonder if something in the userland has changed? |
Hmm, I have a X1 Yoga Gen 8 myself and I just fully updated the F39 on it and then did: gst-launch-1.0 v4l2src ! xvimagesink 10 times and the stream started correctly 10 / 10 times. So I wonder is anything special on your setup(s) ? Are you testing with a charger or thunderbolt dock connected maybe? |
|
Interesting. No charger or dock plugged in. What kernel are you running?
What sensor, and what does `sudo dmidecode -s system-product-name` return
out of interest?
…On Mon, Nov 27, 2023 at 10:34 AM Hans de Goede ***@***.***> wrote:
Strangely enough, my X1 Yoga Gen 8 running Fedora 38 seems to have
regressed to the same behavior described in this issue - I had the kernel
pinned to 6.5.5 but the userland was upgrading. I wonder if something in
the userland has changed?
Hmm, I have a X1 Yoga Gen 8 myself and I just fully updated the F39 on it
and then did:
gst-launch-1.0 v4l2src ! xvimagesink
10 times and the stream started correctly 10 / 10 times.
So I wonder is anything special on your setup(s) ?
Are you testing with a charger or thunderbolt dock connected maybe?
—
Reply to this email directly, view it on GitHub
<#187 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACATUUMXVMEVKZ7TOJNPM3YGRUDFAVCNFSM6AAAAAA5TDSMNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRXGU3DQOJRGQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
lukemarsden.net
***@***.***
@lmarsden <https://twitter.com/lmarsden>
|
|
kernel: 6.5.8-300.fc39.x86_64 |
|
I'm using the new akmod which I linked to above: kmod-intel-ipu6-6.5.8-300.fc39.x86_64-0.0-11.20231115git067270f.fc39.x86_64 |
|
Are you perhaps using the laptop(s) near some electrical appliance which may be giving of electro-magnetic interference? Have you tried using the laptop in some other place, e.g. another room ? |
|
Interesting, my dmidecode is 21HQCTO1WW on the Yoga. I wonder what the
difference entails...
…On Mon, Nov 27, 2023 at 10:57 AM Hans de Goede ***@***.***> wrote:
I'm using the new akmod which I linked to above:
kmod-intel-ipu6-6.5.8-300.fc39.x86_64-0.0-11.20231115git067270f.fc39.x86_64
—
Reply to this email directly, view it on GitHub
<#187 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACATUW447Q6YEFOBYQHRNTYGRW2TAVCNFSM6AAAAAA5TDSMNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRXGYYDKNZTHE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
lukemarsden.net
***@***.***
@lmarsden <https://twitter.com/lmarsden>
|
|
Yeah I've tested it in various different locations. It also works fine on
Windows.
…On Mon, Nov 27, 2023 at 11:00 AM Hans de Goede ***@***.***> wrote:
Are you perhaps using the laptop(s) near some electrical appliance which
may be giving of electro-magnetic interference? Have you tried using the
laptop in some other place, e.g. another room ?
—
Reply to this email directly, view it on GitHub
<#187 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACATURYUCUPW52INYXR3FTYGRXDJAVCNFSM6AAAAAA5TDSMNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRXGYYDSMZQGA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
lukemarsden.net
***@***.***
@lmarsden <https://twitter.com/lmarsden>
|
|
Super weird - after rebooting the X1 Yoga Gen 8 to install some updates,
the camera's now working 100% of the time.
The X1 Carbon Gen 11 this issue initially relates to is still exhibiting
the described issues though.
…On Mon, Nov 27, 2023 at 11:00 AM Luke Marsden ***@***.***> wrote:
Yeah I've tested it in various different locations. It also works fine on
Windows.
On Mon, Nov 27, 2023 at 11:00 AM Hans de Goede ***@***.***>
wrote:
> Are you perhaps using the laptop(s) near some electrical appliance which
> may be giving of electro-magnetic interference? Have you tried using the
> laptop in some other place, e.g. another room ?
>
> —
> Reply to this email directly, view it on GitHub
> <#187 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AACATURYUCUPW52INYXR3FTYGRXDJAVCNFSM6AAAAAA5TDSMNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRXGYYDSMZQGA>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
--
lukemarsden.net
***@***.***
@lmarsden <https://twitter.com/lmarsden>
--
lukemarsden.net
***@***.***
@lmarsden <https://twitter.com/lmarsden>
|
|
Just for the record, back on the X1 Yoga Gen 8 (ov2740), running: I am getting <30% success rate on launching the camera again today, even after rebooting. |
|
Same issue with kernel 6.6.2-101.fc38.x86_64 and 0.0-10.20230622git8e41080.fc38.x86_64 |
|
On 6.6.2, the newer rpm you shared above fails every time now (after three reboots) the same way: |
|
Seeing same problems but I'm running arch on 6.6.7 |
|
I think I have a similar issue: #204 |
|
Latest status report on freshly installed Fedora 40 codeI'm using this Based on latest master of https://github.com/intel/ipu6-drivers - I noticed #213 landed Includes:
stepsSteps to reproduce / what I did (@jwrdegoede @kupietz please check carefully if this looks correct to you):
Webcam now fails to load with: I've also tried powering off for 10 seconds. Detailed relevant dmesg outputI also copied But we don't seem to be getting that far (no kernel logs from ov2740 module, getting stuck on What am I doing wrong? I'd like to get to the point that we can document, from a fresh install, exactly what to do. That would really help other users. |
|
related? https://bugs.launchpad.net/ubuntu/+source/ivsc-driver/+bug/2061747 although aren't we are using in-tree ivsc above? (it's just this bug reports the same |
Looks all correct to me. Here my dmesg with working camera for comparison: And here with int3472-discrete debugging on ( |
I just copied your instructions (I skipped v4l2-relayd config, but I likely had that from before) and this made my camera work, and it hasn't for a long time! (Dell XPS 9320) So thanks to everyone involved, and a confirm that this setup now works for me! //Edit, spoke too soon, camera still hangs sometimes, a lot better than nothing though! Also haven't debugged this so it might also be a chrome video call+wayland hickup |
|
#187 (comment) is the same. But I haven't been able to successfully port these Fedora recipes to Ubuntu to try the patches suggested here. |
|
When the camera hangs, is there a way to reset it without a reboot? I tried At this point, I just need a solution that works in the interim. Trying to avoid buying and carrying a camera, but that might be a more reliable strategy. EDIT: for the record, I have found that |
|
Another problem seems to be: the driver inhibits the kernel from suspending after resets. E.g. 2024-05-17T23:16:53.621836-04:00 raven kernel: Freezing remaining freezable tasks completed (elapsed 0.002 seconds) So it ends up needing a reboot. |
|
Any update on this from the Intel folks, Lenovo, or anyone? My X1C Gen 11 (21HMCTO1WW) with sensor ov2740 is still unreliable even on latest Ubuntu 24.04 with https://launchpad.net/~oem-solutions-group/+archive/ubuntu/intel-ipu6 (which is now working flawlessly on my Yoga Gen 8) but we know disabling runtime PM on the sensor fixes it (#187 (comment)) Please can we figure out how to fix this? I tried but failed to patch the kernel here. |
|
It can be made to work on a 21HMCTO1WW, which I have. I've done a fair bit of testing and it is not 100% flawless, although it is mostly usable. I still need to restart the relay daemon and occasionally reinstall the module. See the recipes toward the end of the thread in #228 for Ubuntu 24.04. |
|
Thanks for replying @The9Cat! Which recipes in particular, sorry? I believe I am already running the confguration here. I am on 6.8.0-1011-oem. Looks like you additionally added the To clarify, I do occasionally get a video signal, but 70% of the time it just hangs, and once it's hung, sometimes the light stays on until after I reboot. |
|
Got it! Sounds like you are doing everything that needs to be done. A few comments:
This is far from an ideal situation, I'm sure that we both agree. More than a bit frustrating. It should just work. But it can be made to work without rebooting. If you can find a better solution, please LMK!! |
|
Hi, I posted 2 patches that should prevent "stream fails to start" issue: At least those make ov2740 sensor issues I have on my Lenovo laptops gone. Third patch (reading FW version) is for diagnostic: would be useful to know what LJCA firmware is in use. |
|
Thanks @sgruszka ! That is good news and I appreciate the effort. The most recent 6.11 oem kernel image for Ubuntu 24.04 seems to be more stable and less glitchy than previous releases as well. |
I just tested the two patches and they result in the ov2740 being properly initialized on my X1 G11 (21HMCTO1WW). |
|
How did you test this? I'm assuming that this was this a full kernel compile? While I've done this before, I'm not feeling up to task at this point. Any idea how long until these patches are approved, etc. and make it into Ubuntu Mainline? |
|
Also, I have tried @sgruszka's script that features the problem. And I tend to get 28/50 tries consistent with my previous findings in #187 (comment) This is also Lenovo X1 C11 (21HMCTO1WW). I am happy to know that this will be fixed for me at some point. Thanks! |
Actually only ljca module need to be rebuild - if you use kernel compiled from source. But yes, if you use distribution kernel build, then it will be require to build and install kernel from source with the fixes applied.
Once get applied to the upstream, what I hope will happen soon, it should take something between one to four weeks to propagate to stable distribution releases. Distribution also carry some external patches, so you can ask kernel maintainers of your distribution to apply it, before it will propage via upstream. |
|
FYI: I posted never version which include small fix for handling error case, bug I made in the first submission: |
|
@sgruszka Where do I find the latest version of your patches? I'll try applying them to a recent Ubuntu mainstream source... Your v4 from Nov 12 seems to be the lastest, is that right? |
|
I had a small block of time for this, so I build the Ubuntu 6.11.0-oem-1007 source using the V4 patches from @sgruszka. It did not work perfectly, alas. After boot up, the driver spat out many of these messages: I was able to recover by |
This indicate ipu6 isys firmware intialization issue. Not sure how this is related with to ov2740 sensor problems. |
|
@The9Cat you can also try this patch, |
If you are working with security-signed firmware but the IPU device is running in non-secure mode, you may see the firmware open failed. |
And how does a user run the IPU device in secure mode, or alternatively switch to a non-security-signed firmware? |
|
@sgruszka I'm attaching the dmesg file. At the 100s mark, I removed and readded the isys module. I turned off secure boot to use the unsigned kernel, but @bingbucao makes a good point. I'll need to remind myself how (or if?) I can make a signed kernel. Or do you have another suggestion on this? Curiously, after removing and reinstalling the isys module it starts to work. Would that happen if the secure firmware is the issue? |
|
Another possibility is that something is wrong with the IVSC. |
|
I applied the patch that @sgruzka mentioned in #187 (comment) and that eliminated the intel-ipu6-isys messages. But the expected virtual video devices were missing and the v4l2 service could not start as a result. |
@The9Cat please open separate ticket for the issue you have and provide exact steps to reproduce there (including steps of downloading proper source packages, applying patches and compilation) . I'll try to reproduce problem on my laptop. |
I have an X1 Carbon Gen 11 21HMCTO1WW with an ov2740 sensor.
I am running the latest 6.5.5 kernel with the Fedora driver stack from https://hansdegoede.livejournal.com/27276.html
Very often the stream just fails to start streaming. I can run
sudo gst-launch-1.0 icamerasrc ! autovideosink10 times in a row and on average, I will only get a video stream 3/10 times. The webcam works reliably in Windows so I doubt it's a hardware problem. The exact same software stack works more reliably on an X1 Yoga Gen 8 with the same sensor (different hardware configuration though, the Yoga has a i7-1370P CPU and the Carbon has a i7-1365U).dmesg -wwhile doing this shows the following pattern:Here you can see when it worked (on, off with no red errors), and when it didn't (timeouts), from
dmesg -w.Here's the output from running sudo gst-launch-1.0 icamerasrc ! autovideosink > gst-1.txt 2>&1 one time that it didn't work:
https://gist.github.com/lukemarsden/d51e5f438d72f3211c402ea4e9af7cf6
And the journal from that time: sudo journalctl -f > journal-1.txt 2>&1:
https://gist.github.com/lukemarsden/a5022bebdadaa5ae34aab822aff636b9
The only log lines that seem interesting are the timeouts from gstreamer and the:
line from the ipu6 driver. That "stream stop time out" only happens after I hit ctrl+c (twice) on the gstreamer stream because I'm seeing a black video feed.
dmesg |grep ipu6:
https://gist.github.com/lukemarsden/01beb5c00d37e57f71706922aa713832
I followed the instructions at https://hansdegoede.livejournal.com/27276.html
Versions:
Please let me know if you need any more details or logs from my system.
The text was updated successfully, but these errors were encountered: