Audio-Video Data Processing
2019-2020
Professors:
Lect. Dr. Adrian Sterca [forest at cs dot ubbcluj dot ro]
Objectives:
At the end of this course, students should have fundamental knowledge in:
- digital audio and video signal representation and processing
- audio-video codec standards like ISO's MPEG and ITU-T H.26[1234]
- audio-video streaming over best-effort networks
- satellite audio-video broadcasting
- computer vision intro
They should also have expertise in designing and implementing an audio-video
processing system.
Examenul quiz o sa aiba loc in saptamana a 14-a a semestrului, adica JOI, 16 IANUARIE 2020, ora 14.00, sala C310 (Campus).
The quiz exam will take place during the last PDAV course of the semester (week 14 of the semester; 16th January 2020, 14.00 hours, room C310).
Below you can find the number of attendances to the course and the marks you received at each lab so far (a SCS network user/password is required):
Attendances
Cine are cel putin 7 prezente la curs si are media pe laboratoare cel putin 6, nu trebuie sa
vina la examenul quiz - se considera ca acei studenti au nota 10 la quiz. De asemenea, nu trebuie sa vina la examenul quiz
cei care au ales criteriul 'proiect'.
News [24-February-2020]: Note finale PDAV
Daca aveti neclaritati sau observati erori in notare, va rog sa imi scrieti mail.
Course description:
- Introduction
- Introduction to multimedia
- Color spaces
- Analog representation of audio and video signals
- Digital teprezentation of still images. The JPEG compression standard
- Digital representation of audio signal
- Audio and video formats (containers), codecs
- AV containers: .avi, .ogg, .mp4, .vob, .3gp, .mkv
- MPEG-1,2,4
- H.264/MPEG-4 AVC and the SVC extension
- H.265, High Efficiency Video Coding (HEVC)
- VP9
- Streaming and signalling protocols
- RTP/RTCP - Real Time Protocol
- RTSP - Real Time Streaming Protocol
- SDP - Session Description Protocol
- SIP - Session Initiation Protocol
- HTTP Streaming, DASH
- Congestion control algorithms for audio-video applications in "best-effort" networks
- TCP AIMD (Additive Increase Multiplicative Decrease)
- DCCP - Datagram Congestion Control Protocol
- TFRC - TCP-Friendly Rate Control
- UTFRC - Utility-driven TCP-Friendly Rate Control
- Voice over IP
- Audio-video communication in satellite networks
- Satellite communications
- Frequency bands
- DBS - Direct Broadcast Satellite (Direct-To-Home)
- DVB standards: DVB-S (satellite), DVB-T (terrestrial), DVB-C (cable)
- The FFMpeg library
- libavcodec
- libavformat
- libswscale
- libavfilter
- Audio-video for the web
- Audio-Video support in HTML 5
- Real-Time Communication on the web (WebRTC)
- Introduction to Computer Vision
- What is Computer Vision ?
- Feature extraction & feature matching
- Image features: SIFT
- Image features: SURF
- Multimedia QoS in Internet
- QoS terminology
- QoS requirements in "best-effort" networks
- P2P TV
- Peer-2-Peer networks
- DHT - Distributed Hash Tables
- Applications
Course Requirements (grading criteria):
There are 3 possible grading criteria and each student is allowed to choose the one that suits him best:
- Project: Students should develop a project related to
audio-video data processing either individually or in small teams of 2 persons. Students must choose
the project theme in the beginning of the semester and they must present the project at the end of the semester.
During the semester, they must also show work progress and intermediate versions of the project at the labs.
The possible grades a student can receive on the project are: 10, 9 and 4. No other grades are possible.
Examples of project ideas are here: List of possible projects (in romanian).
Please note that the theme of
each project and also its developement must be discussed with me at the laboratory because there are
several ways of implementing a specific task (like the ones mentioned in the list of projects from above), some
of them are very easy (using a library) and some of them are more complicated. Generally, I prefer that you
work on your project not relying on an additional external library, but rather do the low-level stuff
yourself.
- Labs + Quiz test: Students must complete 4 laboratory tasks (i.e. 4 labs) during the
semester and at the end of the semester they must take a quiz test. The lab tasks require the students to build a part of a video codec
and are detailed here: Laboratory tasks.
The final grade is formed like this: averageLabGrade*0.8 + quizGrade*0.2
If the student is present to at least 7 courses and averageLabGrade is at least 6, then the quizGrade for this
student is considerred to be 10 and the student does not need to take the quiz test anymore.
- Quiz test: If the student does not get an average lab grade of at least 6 and he/she
does not do a project, he/she must take the quiz test and the grade he/she receives for this quiz test is the final grade. This grade
can not be greater than 6.