Monday, July 19, 2010

Take snapshot from webcam using Silverlight 4 Webcam API

I have Webcam attached to my Laptop, but I’m not able to capture still images from it on Windows 7. I tried Youcam from Cyberlink it’s good but not freely available then I decide to build my own application for capturing images from my webcam.

As we know that Silverlight 4 has Webcam, Microphone and other good APIs built in so I decided to build my application with Silverlight 4, so in this post I’ll show you how you can capture images from your webcam and save it.

So let’s start by creating new Silverlight Application by opening Visual Studio 2010 or Visual Web Developer 2010 Express which is freely available, click File –> New Project

New Project

In the New Project dialog, select Silverlight in Installed Templates section from the Left pane and select Silverlight Application, as you hit ok button a new dialog appears asking you “if you want to create new ASP.NET Web application or ASP.NET MVC application”. See the screenshot below:

Ask for new ASP.NET Web Application

As shown above the dialog is straight forward and easy to understand, just have settings as shown and hit ok button, you now have Silverlight application ready to work with.

Our application will look like this when completed in the designer:

Silverlight Designer

What are we waiting for let design the application first in the xaml designer. The application UI is very simple you see in the screenshot I have two comboBox on top, one for holding VIDEO devices and second for AUDIO devices available in the machine, we can fill these two comboBox within our code, in the center of the UI you see white large box which is for displaying our Webcam live feed, I add three buttons for different purposes, “Start Camera” is used to enable webcam, this same button is also used to disable webcam, “Capture Image” is used to capture current frame from the live webcam feed which is in raw format as WriteableBitmap, finally last button “Save Image” is used to save captured frame in jpg format. On the bottom of the application there is a ListBox which holds our captured raw frames from the webcam, which is bind to ObseravableCollection<WriteableBitmap>.

To design exactly as shown above, just copy and paste the following xaml markup in your MainPage.xaml file

<UserControl x:Class="SilverlightDemo.MainPage"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    mc:Ignorable="d"
    d:DesignHeight="600" d:DesignWidth="600">

    <Grid x:Name="LayoutRoot" Background="#FF333333">
        <StackPanel HorizontalAlignment="Center" VerticalAlignment="Center">
            <TextBlock Foreground="White" FontSize="18" HorizontalAlignment="Center">Silverlight 4 Webcam and Microphone Demo</TextBlock>
            <Grid Width="500">
                <Grid.RowDefinitions>
                    <RowDefinition />
                    <RowDefinition />
                    <RowDefinition />
                    <RowDefinition />
                </Grid.RowDefinitions>
                <Grid.ColumnDefinitions>
                    <ColumnDefinition Width="250" />
                    <ColumnDefinition Width="250" />
                </Grid.ColumnDefinitions>
                <TextBlock Foreground="White" Margin="5" Text="Available VIDEO Sources" Grid.Column="0" Grid.Row="0"></TextBlock>
                <ComboBox x:Name="VideoSources" Grid.Column="0" Grid.Row="1" Margin="5">
                    <ComboBox.ItemTemplate>
                        <DataTemplate>
                            <TextBlock Text="{Binding FriendlyName}"/>
                        </DataTemplate>
                    </ComboBox.ItemTemplate>
                </ComboBox>
                <TextBlock Foreground="White" Margin="5" Text="Available AUDIO Sources" Grid.Column="1" Grid.Row="0"></TextBlock>
                <ComboBox x:Name="AudioSources" Grid.Column="1" Grid.Row="1" Margin="5">
                    <ComboBox.ItemTemplate>
                        <DataTemplate>
                            <TextBlock Text="{Binding FriendlyName}"/>
                        </DataTemplate>
                    </ComboBox.ItemTemplate>
                </ComboBox>
                <Border CornerRadius="8" Grid.ColumnSpan="2" Grid.Row="2" Width="500" Height="400">
                    <Border.Effect>
                        <DropShadowEffect Color="White" Direction="0" ShadowDepth="0" BlurRadius="15"/>
                    </Border.Effect>
                    <Rectangle x:Name="Webcam" Fill="White" Margin="5" Width="500" Height="400"/>
                </Border>
                <StackPanel Orientation="Horizontal" HorizontalAlignment="Center" Grid.ColumnSpan="2" Grid.Row="3">
                    <Button x:Name="StartStopWebcam" Content="Start Camera" Width="100" Height="35" Margin="5" Click="StartStopWebcam_Click"/>
                    <Button x:Name="CaptureWebcam" Content="Capture Image" Width="100" Height="35" Margin="5" Click="CaptureWebcam_Click" />
                    <Button x:Name="SaveImage" Content="Save Image" Width="100" Height="35" Margin="5" Click="SaveImage_Click" />
                </StackPanel>
            </Grid>
            <ScrollViewer Width="500" HorizontalScrollBarVisibility="Auto" VerticalScrollBarVisibility="Hidden">
                <ListBox x:Name="Snapshots">
                    <ListBox.ItemTemplate>
                        <DataTemplate>
                            <Image Source="{Binding}" Margin="5" Stretch="UniformToFill" Height="70"/>
                        </DataTemplate>
                    </ListBox.ItemTemplate>
                    <ListBox.ItemsPanel>
                        <ItemsPanelTemplate>
                            <StackPanel Orientation="Horizontal" HorizontalAlignment="Center" VerticalAlignment="Center"/>
                        </ItemsPanelTemplate>
                    </ListBox.ItemsPanel>
                </ListBox>
            </ScrollViewer>
        </StackPanel>
    </Grid>
</UserControl>

Our UI is complete and we can move on to our code, open the MainPage.xaml.cs file, we first start declaring two global variable like that:

        CaptureSource _captureSource;
        ObservableCollection _images = new ObservableCollection();

we define _captureSource to hold the instance of CaptureSource which is used to interact with AUDIO/VIDEO devices. _images variable is used to holds all captured frames from the webcam and will be the item source for the above said ListBox.

for declaring ObservableCollection<T> type you need to have using System.Collections.ObjectModel;

we also have to declare one public property named SelectedSnapshot, which holds the current captured frame from the live webcam feed and used to save image as jpg:

        private WriteableBitmap selectedSnapshot;
        public WriteableBitmap SelectedSnapshot
        {
            get { return selectedSnapshot; }
            set { selectedSnapshot = value; }
        }

We then add Loaded event handler in the default constructor, see the code below:

        public MainPage()
        {
            InitializeComponent();
            Loaded += new RoutedEventHandler(MainPage_Loaded);
        }

        void MainPage_Loaded(object sender, RoutedEventArgs e)
        {
            // Get list of the VIDEO Sources and bind
            VideoSources.ItemsSource = CaptureDeviceConfiguration.GetAvailableVideoCaptureDevices();

            // Get list of the AUDIO Sources and bind
            AudioSources.ItemsSource = CaptureDeviceConfiguration.GetAvailableAudioCaptureDevices();

            // Select the default devices
            if (VideoSources.Items.Count > 0)
                VideoSources.SelectedIndex = 0;

            if (AudioSources.Items.Count > 0)
                AudioSources.SelectedIndex = 0;

            // Creating CaptureSource
            _captureSource = new CaptureSource();

            // Handle CaptureImageAsync Completed event handler
            _captureSource.CaptureImageCompleted += (s, ev) =>
            {
                ProcessImage(ev.Result);
            };

            // Bind snapshots
            Snapshots.ItemsSource = _images;

            // Disable the capture button, it'll be enabled when capture source is ready
            CaptureWebcam.IsEnabled = false;
            SaveImage.IsEnabled = false;
        }

above method is very simple, which is used to initialize some of our application parts, we first set the item source of AUDIO/VIDEO comboBox, which are filled by getting all the devices available in the user’s machine. I also create a new CaptureSource instance, after that I handled CaptureImageCompleted event handler which is fired after capturing raw image by calling CaptureSource’s CaptureImageAsync() method. Finally I bind the ListBox to captured frames collection by setting it’s ItemSource property to _images.

In the above code you might think of ProcessImage(ev.Result) method, I have this method for processing raw captured frame, the code in that method is simple and does nothing but add the captured frame to _images collection and set the SelectedSnapshot property to the current captured frame.

        private void ProcessImage(WriteableBitmap bitmap)
        {
            _images.Add(bitmap);
            SelectedSnapshot = bitmap;
            SaveImage.IsEnabled = true;
        }

Move on to the “Start Camera” button click event, first I check the State property of the CaptureSource instance if it’s not started then I set the AUDIO/VIDEO Device to the CaptureSource instance from the available devices then I declared a variable of type VideoBrush which is used to paint the video content and set it’s source to CaptureSource instance and finally set it to the Fill property of Rectangle which paints the live video feed from the webcam.

To access the AUDIO/VIDEO devices on the user’s machine we need to have permission, we can request the user’s permission by calling the following code:

                        // Request user permission
                        if (CaptureDeviceConfiguration.AllowedDeviceAccess || CaptureDeviceConfiguration.RequestDeviceAccess())
                        {
                            if (_captureSource.VideoCaptureDevice != null)
                            {
                                _captureSource.Start();
                                StartStopWebcam.Content = "Stop Camera";
                                CaptureWebcam.IsEnabled = true;
                            }
                        }

When the above code is executed a dialog window is popup and asking for your permission to access the AUDIO/VIDEO devices, See screenshot below:

Permission Popup

By clicking Yes you allowed application to access AUDIO/VIDEO devices, just after having permission we can see the live webcam feed in the center of the UI, Let’s see how can I look?

Running Application

Ok once we started capturing we can stop it with the same button which stops capturing from the webcam. you noticed that “Capture Image” button is now activated after starting the webcam which takes snapshot from the current frame and displayed it in ListBox below, “Capture Image” button click event has the following code:

            // Verify the device is started
            if (_captureSource.VideoCaptureDevice != null && _captureSource.State == CaptureState.Started)
            {
                // Capture the current frame
                _captureSource.CaptureImageAsync();
            }
        }

In the code above I just checked the State property of the CaptureSource instance which must be Started if so then I called the CaptureImageAsync() method, which initiates an asynchronous image capture request. After taking snapshot “Save Image” button is activated which is used to save captured raw bitmap in jpg format on the local storage media, there is no built in function to do so that’s why I’m taking advantage of .NET Image Tools which have fine API for working with raw images. I have included the required binaries in the project itself so you don’t have to download it, but you can if you want complete set like documentation and other encoders [PNG, BMP, GIF].

To work with Image Tools we first need to reference the required assemblies which are listed below:

  • ICSharpCode.SharpZipLib.Silverlight.dll
  • ImageTools.dll
  • ImageTools.IO.Jpeg.dll
  • ImageTools.Utils.dll

Above assemblies are included in the project within the Lib folder which is available for download see the link below. Ok we now ready to save our webcam snapshots, first we need to have using statements in the top of the code file like following:

using ImageTools;
using ImageTools.IO.Jpeg;

In “Save Image” button click event I have following code which actually save the raw image in jpg format:

            // Gets the current captured raw bitmap
            var rawImage = SelectedSnapshot;

            // If image is selected in the Snapshots ListBox then set it as a save target
            if (Snapshots.SelectedItem != null)
                rawImage = (WriteableBitmap)Snapshots.SelectedItem;

            if (rawImage != null)
            {
                // Init the Save File Dialog
                SaveFileDialog saveDialog = new SaveFileDialog();
                saveDialog.Filter = "JPG Files (*.jpg, *.jpeg)|*.jpg;*.jpeg|All Files (*.*)|*.*";
                saveDialog.DefaultExt = ".jpg";
                saveDialog.FilterIndex = 1;

                // Show save dialog to the user
                if ((bool)saveDialog.ShowDialog())
                {
                    using (Stream stream = saveDialog.OpenFile())
                    {
                        // Convert raw captured bitmap to the image that Image Tools understand with the extension method
                        var image = rawImage.ToImage();
                        // Declare jpeg encoder
                        var encoder = new JpegEncoder();
                        // Set the image quality
                        encoder.Quality = 90;
                        // Finally encode raw bitmap and save it as a jpg image
                        encoder.Encode(image, stream);
                        // Close the stream
                        stream.Close();
                    }
                }
            }

The code above again is very simple and I commented it well so I don’t think to described it for you, if you any problem then discuss it in the comments. I am done here, hope you like it.

Finally, you can download the full source code below:

31 comments:

  1. Hi, kshahnawaz. when i would like to save the photo. i keep get this problem 'The type initializer for 'ImageTools.ImageExtensions' threw an exception.'
    any idea what is the problem? or i do some mistake?

    ReplyDelete
  2. Hi,
    I try your project for capture part its work fine. however i keep meet this problem when i try to save the picture
    "The type initializer for 'ImageTools.ImageExtensions' threw an exception."
    any idea what is the problem or i do some mistake.

    Thanks for share your project. nice project.

    ReplyDelete
  3. Hi Halo,

    Yes there was a problem on saving captured image, I just viewed the error stack trace and found that the project was missing PNG encoder, I added it in the project's lib folder and then it worked fine. Please download the project again, sorry for any inconvenience.

    Hope this works!

    ReplyDelete
  4. Hi Shahnawaz,
    Actually my user want only to view the captured image on other page not to save by file dialog.Hence image will be saved but not by the user.I am zero in SL means i am a beginner in SL. The one way could be save it into DB,it will not be efficient but it will work.
    And one thing more have u ever worked out with video chat?
    if yes then pls provide me link.
    But at all ur work is rocking.It must be appreciated.
    Thanks

    ReplyDelete
  5. Hello ,
    Vishal here,
    can u tell me namespace or refrence for CaptureSource?
    I am using 2010 version of VS and SL version 3.
    CaptureSource is producing error on run time.

    ReplyDelete
  6. FYI, Silverlight Webcam API has been introduced in SL 4, you must install SL 4 to find CaptureSource.

    Hope it helps!

    ReplyDelete
  7. If you want to show captured image on any other page or area of the app. then you can do it easily, as we captured image as WriteableBitmap [raw image] and you can hold its value [in global variable] and apply on the control which accepts WriteableBitmap as it's property value, for saving it in DB yes you can do it, all you need to do is convert WriteableBitmap object in binary format [you have to bingoogle it] and save in DB.

    No, I didn't ever tried Video Chat programming.

    Hope it helps!

    ReplyDelete
  8. Sorry,i tried a lot but i couldn't find the solution.

    ReplyDelete
  9. Hello Mr Khan.
    Really nice post Appreciate
    I just stuck in my web application because of one problem.
    I am also using the same methodology but i would like to save the image directly on server [In Specific Folder] Or as a bytes of stream in DB ?
    Can you tell me is it possible to do ?

    Thanks
    Pushkar

    ReplyDelete
  10. Hi Shahnawaz,
    ur work is rocking. But i want to work with video chat?
    plz help me .

    ReplyDelete
  11. dont you think this is a costly solution , because the end user would not like to download any extra sw in order to use your website ?

    http://www.uobabylon.edu.iq

    ReplyDelete
  12. Yep! I agree, it will requires Silverlight plugin for web browsers, but I developed this app for Out-of-browser experience for my self as I didn't have any native Win-7 app to capture image(s) from my Webcam.

    I hope this would not required with the help of HTML 5 :-)

    Although, in earlier days you also required Adobe-Flash plugin on youtube to view videos, however youtube now uses HTML 5 native video/audio elements, all you need to upgrade your web browser my dear.

    hope this helps!

    ReplyDelete
  13.  i want save it without dialogbox....

    ReplyDelete
  14. hi vishal if you want namespace for capture source it is system.windows.dll file which comes only in silverlight 4

    ReplyDelete
  15. Ibraheem Osama MohamedApril 11, 2012 at 1:43 AM

    Very Nice Blog :) Thanks

    ReplyDelete
  16. dear I have a question that i want to load or embeddd a c# winform in asp.net is that possible.plz reply

    ReplyDelete
  17. Hello sir,
    I need ur help... I want to save the captured image in the folder at server side..
    kindly plz provide me with the solution..
    Thanks in advance...

    ReplyDelete
  18. Hello sir,
    i need ur help...
    i want to save the captured image into the folder located at server side on clicking the Save button..
    can u plz provide me with the code...
    thanx in advance!!!

    ReplyDelete
  19. Hello Neha,


    I've just worked around your requirements and you can download the latest code from my skydrive account:


    http://sdrv.ms/NncE4r



    Thanks!

    --Shahnawaz

    ReplyDelete
  20. Firstly congratulations solution!

    Now I'd like your help to adapt to my project, two adaptations.

    1 - Save as jpg without using "SaveFileDialog SaveDialog = new SaveFileDialog ()" right now as it does in. PNG.

    2 - Name the file to be saved not as NewGuid (), but with this ID in a HttpContext.Current.Session ("IDCLIENT") on the server.

    Hope I'm not asking for much, am beginner silverlight and would greatly appreciate your valuable help.
    thank you
    Edgar

    ReplyDelete
  21. Firstly congratulations solution!

    Now I'd like your help to adapt to my project, two adaptations.

    1 - Save as jpg without using "SaveFileDialog SaveDialog = new SaveFileDialog ()" right now as it does in. PNG.

    2 - Name the file to be saved not as NewGuid (), but with this ID in a HttpContext.Current.Session ("IDCLIENT") on the server.

    Hope I'm not asking for much, am beginner silverlight and would greatly appreciate your valuable help.
    thank you
    Edgar

    ReplyDelete
  22. Firstly congratulations solution!
    Now I'd like your help to adapt to my project, two adaptations.
    1 - Save as jpg without using "SaveFileDialog SaveDialog = new SaveFileDialog ()" right now as it does in. PNG.
    2 - Name the file to be saved not as NewGuid (), but with this ID in a HttpContext.Current.Session ("IDCLIENT") on the server.


    Hope I'm not asking for much, am beginner silverlight and would greatly appreciate your valuable help.
    thank you
    Edgar

    ReplyDelete
  23. Firstly congratulations solution!


    Now I'd like your help to adapt to my project, two adaptations.


    1 - Save as jpg without using "SaveFileDialog SaveDialog = new SaveFileDialog ()" right now as it does in. PNG.


    2 - Name the file to be saved not as NewGuid (), but with this ID in a HttpContext.Current.Session ("IDCLIENT") on the server.


    Hope I'm not asking for much, am beginner silverlight and would greatly appreciate your valuable help.
    thank you
    Edgar

    ReplyDelete
  24. Edgar,


    I already done what you need and share the link within the comment below, I'm again sharing it with you, please open the solution in VS and make changes as you need.
    http://sdrv.ms/NncE4r
    Hope this helps!

    ReplyDelete
  25. Could u Help please?

    I took your code and putted into my program. It's can not save in the folder.... say's tha "it does not exist".

    In this Method:

    public string EndSaveImageOnServer(System.IAsyncResult result) {
    object[] _args = new object[0];
    string _result = ((string)(base.EndInvoke("SaveImageOnServer", _args, result)));
    return _result;
    }

    The remote server return: Not Found

    Thank you

    ReplyDelete
  26. Hi, I have implemented your code in my project, there i have struck how to add service reference in my project please let me know....

    ReplyDelete
  27. hi i am trying to put your code. And i got exception like Cross domain policy not accessing how to access this cross domain policy please help in that... thanking you advance

    ReplyDelete
  28. Hello sir,
    This helped me a lot thank you for that. My query is that i want record a vedio instead of image can you please help me. Give me the rly to my mailid "Pabba.mahesh60@gmail.com" Thank You.

    ReplyDelete
  29. hi

    It was Very helpful article ..
    i want to save images with unique ID 's in savefiledialogue box Is It Posible?

    ReplyDelete
  30. Hi Hello,

    Your application looks too good and helpful.Can you send me this project to my email.

    kaaju13@gmail.com


    Thank you..:)

    ReplyDelete

Thanks for posting valuable comments, your comments will be reviewed before publishing.