Upload
others
View
12
Download
0
Embed Size (px)
Citation preview
嵌入式计算机系统
Lecture #8
MeeGo Multimedia
内容来自于meego.com以及MeeGo相关公开教程
Outline
• MeeGo Multimedia Module Overview
• Qt Mobility Multimedia
• GStreamer Framework
• APIs in Qt Mobility Multimedia
Multimedia in MeeGo Architecture
Multimedia in MeeGo Architecture
Media Stack Architecture
5
Media Application
Qt-Mobility MeeGo Touch
framework
GStreamer Qt
Video Decoder
Video Encoder Graphics
SST audio engine
HW/Driver
Other core modules
Core OS
libva
Multimedia in MeeGo Platform API
• GStreamer
• PulseAudio
Multimedia in MeeGo API
Qt Mobility Overview • Qt Mobility Project provides desirable functions for
mobile platform
• Qt Mobility covers a collection of APIs or frameworks
Bearer Management Contacts Location Message Multimedia
Publish and subscribe
Qt Service Framework
Sensors API
System Info …
Qt Mobility Multimedia Module • QtMobility’s media library will
• Easily create applications with system media capabilities • Cover the items of most direct interest to developers
• playback • recording • playlist • management • metadata • radio • media editing • TV • transcoding • streaming
Qt Mobility Multimedia Module
• Advantage: • Implement fundamental Multimedia functions with
minimal code • A great deal of flexibility with the media source • Many different codecs are supported 'out of the box'
Qt Mobility Multimedia Module
• High Level APIS • Encapsulate the ability to manipulate media stream at varying level
without detailed knowledge of media subsystem
QT Mobility Multimedia Service
QT Mobility Multimedia High Level APIs
Playback List
Audio Video
Recording Playback Radio .. Camera
QtMobility Multimedia Module
• Service - access to low-level of Multimedia function • QAudio*
• QAudioDeviceInfo, QAudioInput, QAudioOutput • Low level cross platform audio API • Low latency notifications, simple game, audio data processing/visualisation.
• QVideo* • QAbstractVideoBuffer, QVideoFrame, QabstractVideoSurface • Low level tools for passing video frames. • Abstraction for video buffers and surfaces
QtMobility Multimedia Module
• QT MMF supports rich backend: • GStreamer • Quicktime • Directshow • …
QT MMF
GStreamer Service Plugin
QGstreamerService
QuickTimeService Plugin
QuickTimeService MyService
What is GStreamer
• A framework for creating media applications • Pipeline based • Not a player • One of the most obvious uses is to build a media player
• Main advantage • Pluggable elements could be mixed to form arbitrary
pipelines to support various kinds of video/audio codecs • Unified API for user to create pipelines and media players
GStreamer Framework
Concepts in GStreamer
• Element
• Pad
• Bin
• Pipeline
• Communication Mechanism
Concepts in GStreamer-element • Element
• Element is the most important object in GStreamer, it is derived from GstObject
• Create a chain of elements linked together and let data flow through
this chain of elements
Concepts in GStreamer-element
• The way to create an element
GstElement *element;
element = gst_element_factory_make(“fakesrc”, “source”);
if(!element) {
g_print(“Failed to create element\n”);
return -1;
}
Concepts in GStreamer-element
• Using an element as a GObject GstElement *element; gchar *name; element = gst_element_factory_make(“fakesrc”, “source”); if(!element) { g_print(“Failed to create element\n”); return -1; }
//get the name of element g_object_get(G_OBJECT(element), “name”, &name, NULL); g_print(“The name of element is %s.\n”,name); g_free(name);
Concepts in GStreamer-element
• Linking elements GstElement *pipeline; GstElement *source, *filter, *sink; pipeline = gst_pipeline_new(“my-pipeline”); source = gst_element_factory_make(“fakesrc”, “source”); filter = gst_element_factory_make(“identity”, “filter”); sink = gst_element_factory_make(“fakesink”, “sink”); gst_bin_add_many(GST_BIN(pipeline), source, filter, sink, NULL); if(!gst_element_link_many(source, filter, sink, NULL)) { g_warning(“Failed to link element!”); }
Concepts in GStreamer-element
• Element States • GST_STATE_NULL • GST_STATE_READY • GST_STATE_PAUSED • GST_STATE_PLAYING
• Setting the element state • gst_element_set_state() • gst_element_sync_state_with_parent()
Concepts in GStreamer-pad
• Pad • Pad is used to negotiate links and data flow between
elements in GStreamer • Pad could be sink pad or source pad • Pad can restrict the type of data that flows through it
Concepts in GStreamer-pad • Pad GObject +----GstObject +----GstElement +----ffdemux_ogg Pad Templates: SRC template: 'video_%02d' Availability: Sometimes Capabilities: ANY SRC template: 'audio_%02d' Availability: Sometimes Capabilities: ANY SINK template: 'sink' Availability: Always Capabilities: application/ogg
GObject +----GstObject +----GstElement +----GstMpeg2dec Pad Templates: SRC template: 'src' Availability: Always Capabilities: video/x-raw-yuv format: { YV12, I420, Y42B } width: [ 16, 4096 ] height: [ 16, 4096 ] framerate: [ 0/1, 2147483647/1 ] SINK template: 'sink' Availability: Always Capabilities: video/mpeg mpegversion: [ 1, 2 ] systemstream: false
Concepts in GStreamer-pad • An example of using Dynamic pads static void cb_new_pad(GstElement *element, GstPad *pad, gpointer data) {
gchar *name; name = gst_pad_get_name(pad); //setup a new pad link for the newly create pad …
} GstElement *demux; demux = gst_element_factory_make(“oggdemux”, “demuxer”); //listen for newly create pads g_signal_connect(demux, “pad-added”, G_CALLBACK(cb_new_pad), NULL);
Concepts in GStreamer-Bin&Pipeline
• Bin & Pipeline • A Bin is a container for a collection of elements. E.g. playbin • A pipeline is a special subtype of a bin that allows execution of all of
its contained child elements. • A pipeline is a top-level bin. As you set it to PAUSED or PLAYING
state, data flow will start and media processing will take place. • GstPipeline is derived from GstBin
Concepts in GStreamer-Bin&Pipeline
• Pipeline example - a simple ogg player
Concepts in GStreamer-Bin&Pipeline
• Creating a bin and add elements GstElement *bin, pipeline, *source, sink; //create pipeline pipeline = gst_pipeline_new(“my_pipeline”); //create bin bin = gst_bin_new(“my_bin”); source = gst_element_factory_make(“fakesrc”, “ source”); sink = gst_element_factory_make(“fakesink”, “sink”); //add elements to the bin gst_bin_add_many(GST_BIN(bin), source, sink, NULL); //add the bin to the pipeline gst_bin_add(GST_BIN(pipeline), bin);
Concepts in GStreamer-Communications
• Mechanism for communication and data exchange between the application and the pipeline • buffers: sent between elements in the pipeline • events: sent between elements or from the application to
element • messages: posted by element and collected by
application • queries: application or element queries information from
other element
Concepts in GStreamer-Communications
• Different communications flows in GStreamer pipeline
Concepts in GStreamer-Communications
• Example of using bus static GMainLoop *loop; static gboolean my_bus_callback(Gbus *bus, GstMessage *message,
gpointer data) { switch(GST_MESSAGE_TYPE(message)) { case GST_MESSAGE_ERROR: { … break; } case GST_MESSAGE_EOS: { … break; } default: break; }
}
Concepts in GStreamer-Communications
• Example of using bus
GstElement *pipeline; GstBus *bus; pipeline = gst_pipeline_new(“my_pipeline”); bus = gst_pipeline_get_bus(GST_PIPELINE(pipeline)); gst_bus_add_watch(bus, my_bus_callback, NULL); loop = g_main_loop_new(NULL, FALSE); g_main_loop_run(loop);
Tools in GStreamer • gst-inspect
• Print info about a GStreamer plugin or element • Example: gst-inspect oggdemux
• gst-launch • Build and run a GStreamer pipeline • Example: gst-launch filesrc location=/home/child.ogg ! mad !
alsasink
Example of video player
Player->play() QServicePlugin
QGstremerService
Playbin2
Typefind/autoplug
Demuxer
Decoder
Filter
…
QT MMF API QT MMF Backend GStreamer Base 3rd party plugins
Developing Developed
Multimedia Module API Hierarchy
QObject
QMediaObject
QAudioCaptureSource
QCamera
QMediaImageViewer
QMediaPlayer
QMediaService QMediaControl
QCameraControl
QMediaPlayerControl
QMediaRecorderControl
QMediaPlaylist QMediaRecorder QVideoWidget
Multimedia Module API
• Multimedia module consists of APIs related to • audio • video • image • camera
Multimedia Module API
• Multimedia module consists of APIs related to • audio • video • image • camera
Audio
• Classes related to Audio • QAudioCaptureSource • QAudioDeviceInfo • QAudioEncoderControl • QAudioEncoderSettings • QAudioEndpointSelector • QAudioFormat • QAudioInput • QAudioOutput
QAudioCaptureSource
• Provides an interface to query and select an audio input endpoint
• Get and Set audio input to use
QAudioCaptureSource • Functions in QAudioCaptureSource
• Qlist<QString> QAudioCaptureSource::audioInputs() const Returns a list of available audio inputs • void QAudioCaptureSource::setAudioInput(const Qstring &name) Set the active input to name
QAudioCaptureSource
• Example to use QAudioCaptureSource QAudioCaptureSource *audiosource = new QAudioCaptureSource; //get audio input device names Foreach(const QString &device,audiosource->audioInputs()) {
ui->deviceBox->addItem(device,QVariant(device)) } //set audio input device name audiosource.setAudioInput(boxValue(ui->codecBox).toString()); //set the audiosource to the recorder QMediaRecorder *capture = new QMediaRecorder(audiosource); capture->record();
QAudioEncoderSettings • Provides a set of audio encoder settings int bitRate() Return the bit rate of the compressed audio stream in bits per second void setBitRate(int rate) Set the audio bit rate in bits per second
QString codec() Return the audio codec void setCodec(const QString &codec) Set the audio codec
QtMultimediaKit::EncodingQuality quality() Returns the audio encoding quality void setQuality(QtMultimediaKit::EncodingQuality quality) Set the audio encoding quality
QAudioEncoderSettings
• Example to use QAudioEncoderSettings
QAudioEncoderSettings audioSettings;
audioSettings.setCodec(“audio/mpeg”);
audioSettings.setQuality(QtMultimediaKit::HighQuality);
audioSettings.setChannelCount(2);
aduioSettings.setEncodingMode(QtMultimediaKit::ConstantQualityEncoding);
//set the audio encoder setting when record
QMediaRecorder *capture = new QMediaRecorder(audiosource);
capture.setEncodingSettings(audioSettings);
QMediaRecorder
• Be used for the recording of media content
• Intended to be used with other media object such as QAudioCaptureSource, QRadioTuner
• State of QMediaRecorder • QMediaRecorder::StoppedState • QMediaRecorder:: RecordingState • QMediaRecorder:: PausedState
QMediaRecorder
• Signals in class QMediaRecorder • void durationChanged(qint64 duration) emitted when the duration of the recorded media has
changed • void stateChanged(QMediaRecorder::State state) emitted when a media recorder’s state has changed • void mutedChanged(bool muted) emitted when the muted state has changed • void error(QMediaRecorder::Error error) emitted when an error has occurred
QMediaRecorder • Functions in class QMediaRecorder
• void record() start recording • void stop() stop recording • void pause() pause recording • bool setOutputLocation(const Qurl &location) set the destination location of media content • void setEncodingSettings(const QAudioEncoderSettings &audio) set the audio and video encoder settings
QMediaRecorder • Example of using QMediaRecorder
Examples
• Audioinput
• audiooutput
11-8-18 Embedded Computer Systems 47
Multimedia Module API
• Multimedia module consists of APIs related to • audio • video • image • camera
Video
• Classes related to video • QMediaPlayer • QMediaPlayerList • QVideoWidget
QMediaPlayer
• Properties in QMediaPlayer • duration: the duration of the current media • muted: the muted state of the current media • playbackRate: the playback rate of the current media • playlist: the media playlist being used by the player
object • position: the playback position of the current media • volume: the current playback volume
QMediaPlayer
• Properties in QMediaPlayer • media: the active media source being used by the
player object • state: the media player’s playback state QMediaPlayer::StoppedState QMediaPlayer::PlayingState QMediaPlayer::PausedState
QMediaPlayer
• Functions in QMediaPlayer • void play() start or resuming playing the current source • void pause() pause playing the current source • void stop() stop playing and reset the play position to the
beginning
QMediaPlayer
• Example to use QMediaPlayer
QMediaPlayer *player = new QMediaPlayer;
player->setMedia(QUrl::fromLocalFile(“/home/song.mp3”));
player->setVolume(50);
player->play();
QMediaPlaylist
• Provide a list of media content to play
• Used with other media object like QMediaPlayer or QMediaImageViewer
QMediaPlaylist
• Functions in QMediaPlaylist • bool addMedia(const QMediaContent &content) append the media content to the playlist • bool insertMedia(int pos, const QMediaContent &content) insert the media content to the playlist at position pos • int mediaCount() const return the number of items in the playlist • void next() advanced to the next media content in playlist • void previous() return to the previous media content in playlist
QVideoWidget
• Provides a widget which presents video produced by a media object
QVideoWidget
• Properties in QVideoWidget • brightness: int the brightness of displayed video ranging between -100 and 100 • contrast: int the contrast of displayed video ranging between -100 and 100 • hue: int the hue of displayed video ranging between -100 and 100 • saturation: int the saturation of displayed video ranging between -100 and 100 • fullScreen:bool whether video display is confined to a window or is fullScreen
Example
• Example to use QMediaPlayer with QMediaPlaylist and QVideoWidget
playlist = new QMediaPlaylist; playlist->addMedia(Qurl(“http://example.com/movie1.mp4”)); playlist->addMedia(Qurl(“http://example.com/movie2.mp4”));
player = new QMediaPlayer; player->setPlayList(playlist);
videoWidget = new QVideoWidget; player->setVideoOutput(videoWidget); videoWidget->show();
player->play();
Example
• Videowidget
11-8-18 Embedded Computer Systems 59
Multimedia Module API
• Multimedia module consists of APIs related to • audio • video • Image • camera
Image
• Classes related to Image: • QMediaImageViewer
QMediaImageViewer
• Properties in QMediaImageViewer • elapsedTime: int time elapsed since the current image was loaded • timeout: int time that an image is displayed for before moving to
the next image • state: QMediaImageViewer::State playlist control state of a slide show
QMediaImageViewer
• Example of using QMediaImageViewer with QVideoWidget
viewer = new QMediaImageViewer;
videoWidget = new QVideoWidget;
viewer->bind(videoWidget);
videoWidget->show();
QMediaImageViewer
• Example of using QMediaImageViewer with QMediaPlaylist
playlist = new QMediaPlaylist; playlist->setPlaybackMode(QMediaPlaylist::Loop); playlist->addMedia(image1); playlist->addMedia(image2); playlist->addMedia(image3); view->setPlaylist(playlist); view->setTimeout(5000); view->show();
Multimedia Module API
• Multimedia module consists of APIs related to • audio • video • Image • camera
Camera
• Provides a framework to use Camera device features when supported by the platform • Take still photo • Take video • Basic settings including flash mode, zoom level etc
• Classes related to Camera • QCamera • QCameraViewfinder • QCameraImageCapture
Camera
• using camera to capture an image
QCamera *camera = new QCamera; QCameraViewfinder *viewFinder = new QCameraViewfinder(); viewFinder->show(); camera->setViewfinder(viewFinder); QCameraImageCapture *imageCapture = new
QCameraImageCapture(camera); camera->setCaptureMode(QCamera::CaptureStillImage); camera->searchAndLock; imageCapture->capture(); camera->unlock();
Camera
• using camera to record video
QCamera *camera = new QCamera;
QMediaRecorder *mediaRecorder = new QMediaRecorder(camera);
camera->setCaptureMode(QCamera::CaptureVideo);
camera->start();
mediaRecorder->record();