Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Different behavior when onaddtrack is fired for element.captureStream #100

Open
bc-lee opened this issue Jun 8, 2024 · 2 comments
Open

Different behavior when onaddtrack is fired for element.captureStream #100

bc-lee opened this issue Jun 8, 2024 · 2 comments

Comments

@bc-lee
Copy link

bc-lee commented Jun 8, 2024

Consider the following code:

<!DOCTYPE html>
<html>
<head>
    <title>captureStream for Video Element</title>
</head>
<body>

<video id="video"></video>
<button onclick="streamFromVideo()">Stream from Video Element</button>

<script>
  function streamFromVideo() {
    const video = document.getElementById('video');
    video.src = './test-av-384k-44100Hz-1ch-320x240-30fps-10kfr.webm';
    video.onerror = (error) => {
      console.error('Error:', error);
    };
    if (typeof video.captureStream !== 'function' && typeof video.mozCaptureStream === 'function') {
      HTMLVideoElement.prototype.captureStream = HTMLVideoElement.prototype.mozCaptureStream;
    }

    const stream = video.captureStream();
    stream.onaddtrack = () => {
      console.log('onAddTrack');
      const tracks = stream.getTracks();
      for (const track of tracks) {
        console.log('Track readyState:', track.readyState);
        console.log('Track kind:', track.kind);
      }
      console.log('Video tracks:', stream.getVideoTracks().length);
      console.log('Audio tracks:', stream.getAudioTracks().length);
    };
  }
</script>
</body>

The test video file test-av-384k-44100Hz-1ch-320x240-30fps-10kfr.webm has both audio and video tracks. This code is almost identical to the one in the web platform test.

Below are the logs from the code above:

Chrome 125.0.6422.141:

onAddTrack
Track readyState: live
Track kind: audio
Track readyState: live
Track kind: video
Video tracks: 1
Audio tracks: 1
onAddTrack
Track readyState: live
Track kind: audio
Track readyState: live
Track kind: video
Video tracks: 1
Audio tracks: 1

Firefox 126.0:

onAddTrack
Track readyState: live
Track kind: audio
Video tracks: 0
Audio tracks: 1
onAddTrack
Track readyState: live
Track kind: audio
Track readyState: live
Track kind: video
Video tracks: 1
Audio tracks: 1

In Chrome, the onaddtrack is fired twice, however even with the first event, both audio and video tracks are present in the stream. In Firefox, the onaddtrack is fired twice, but the first event only has the audio track, and the second event has both audio and video tracks.

I wonder which behavior is correct. The current web platform test assumes that the behavior in Chrome is correct (it checks only the first event), but it may be an implementation detail of Chrome.
@steely-glint
Copy link

I think you should probably look at the event.
The code above is looking at the state of the stream.

stream.onaddtrack = (ev) => { console.log('onAddTrack kind: '+ev.track.kind );

@Pehrsons
Copy link
Contributor

I would agree the spec can be clearer on this.

I think the question boils down to whether the events fire sync or async in the text:

If the selected VideoTrack or enabled AudioTracks for the media element change, a addtrack event with a new MediaStreamTrack is generated for each track that was not previously selected or enabled; and a removetrack events is generated for each track that ceases to be selected or enabled. A MediaStreamTrack MUST end prior to being removed from the MediaStream.

I'll also note that the addtrack event on AudioTrackList and VideoTrackList in the media element spec is fired sync, while the change event on the same objects is fired async.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@steely-glint @Pehrsons @bc-lee