When H264Packet
is created with is_avc: true
, all payloads that are depacketized are in the AVC format with the length prepending each nalu. This is correct except for SPS and PPS nalus, as AVC encoding expects these to be configured in an AVCDecoderConfigurationRecord
.
Is this something you'd be open to a PR for to have that built into H264Packet?
The code I currently use for this is:
pub fn convert_to_avc_decoder_config_record(
sps_records: &Vec<Bytes>,
pps_records: &Vec<Bytes>,
) -> Option<Bytes> {
if sps_records.is_empty() || pps_records.is_empty() {
return None;
}
let mut bytes = BytesMut::new();
bytes.put_u8(1); // version
bytes.put_u8(sps_records[0][1]); // profile
bytes.put_u8(sps_records[0][2]); // compatibility
bytes.put_u8(sps_records[0][3]); // level
bytes.put_u8(0xFC | 3); // reserved (6 bits), nalu length size - 1 (2 bits) ????
bytes.put_u8(0xE0 | (sps_records.len() as u8)); // reserved (3 bits), num of SPS (5 bits)
for sps in sps_records {
bytes.put_u16(sps.len() as u16);
bytes.extend_from_slice(&sps);
}
bytes.put_u8(pps_records.len() as u8);
for pps in pps_records {
bytes.put_u16(pps.len() as u16);
bytes.extend_from_slice(&pps);
}
Some(bytes.freeze())
}
At first glance it seems like a path to implementing this is for the H264Packet
type to store an Option<Bytes>
field for both a cached sps and cached pps. When it caches both a sps and pps then that depacketize()
call would return the AVCDecoderConfigurationRecord
bytes. Calls to depacketize()
calls before receiving the sps and pps records would always return an empty payload.
I'm not totally clear how this gets handled when new SPS and PPS records come down from RTP, especially since AVC based systems I'm working with (RTMP) seem to ignore later sequence headers, so I'm not sure if H264Packet
needs to contain a collection of seen SPS and PPS headers, or only the first ones.