Saturday, July 13, 2013

Detecting faces in your webcam stream with OpenCV/java and displaying back the results

In this tutorial for java we detected the faces on a file picture and display a square around it.

In this series of tutorials we learnt:
  1. Creating windows in java and drawing in them
  2. Capture the camera stream with OpenCV and display using Java libraries (awt/swing).
In this tutorial for C++ we detected the faces and eyes on a input stream from a camera.

Now we want to do the same thing but in Java. Basically translate this C++/OpenCV tutorial into Java/OpenCV.

Got stucked in several places. Even placed a stackoverflow post. Here is the final code (I didn't do the eyes detection but that it is pretty straightforward...). This is actually the 2nd post on the code. This one is an optimized version where I corrected one error (see comments below), changed the way to get the mat (from OpenCV) into BufferedImage (from Java) using the comment below (thank you!), and defined more meaningful objects... The initial older version is further below, for reference... By the way, check also this post benchmarking different methods on how to go from Mat to BufferedImage.
 /*  
  * Captures the camera stream with OpenCV  
  * Search for the faces  
  * Display a circle around the faces using Java
  */  
 import java.awt.*;  
 import java.awt.image.BufferedImage;  
 import java.awt.image.DataBufferByte;  
 import javax.swing.*;  
 import org.opencv.core.Core;  
 import org.opencv.core.Mat;  
 import org.opencv.core.MatOfRect;  
 import org.opencv.core.Point;  
 import org.opencv.core.Rect;  
 import org.opencv.core.Scalar;  
 import org.opencv.core.Size;  
 import org.opencv.highgui.VideoCapture;  
 import org.opencv.imgproc.Imgproc;  
 import org.opencv.objdetect.CascadeClassifier;  
 class My_Panel extends JPanel{  
      private static final long serialVersionUID = 1L;  
      private BufferedImage image;  
      // Create a constructor method  
      public My_Panel(){  
           super();   
      }  
      /**  
       * Converts/writes a Mat into a BufferedImage.  
       *   
       * @param matrix Mat of type CV_8UC3 or CV_8UC1  
       * @return BufferedImage of type TYPE_3BYTE_BGR or TYPE_BYTE_GRAY  
       */  
      public boolean MatToBufferedImage(Mat matBGR){  
           long startTime = System.nanoTime();  
           int width = matBGR.width(), height = matBGR.height(), channels = matBGR.channels() ;  
           byte[] sourcePixels = new byte[width * height * channels];  
           matBGR.get(0, 0, sourcePixels);  
           // create new image and get reference to backing data  
           image = new BufferedImage(width, height, BufferedImage.TYPE_3BYTE_BGR);  
           final byte[] targetPixels = ((DataBufferByte) image.getRaster().getDataBuffer()).getData();  
           System.arraycopy(sourcePixels, 0, targetPixels, 0, sourcePixels.length);  
           long endTime = System.nanoTime();  
           System.out.println(String.format("Elapsed time: %.2f ms", (float)(endTime - startTime)/1000000));  
           return true;  
      }  
      public void paintComponent(Graphics g){  
           super.paintComponent(g);   
           if (this.image==null) return;  
            g.drawImage(this.image,10,10,this.image.getWidth(),this.image.getHeight(), null);  
           //g.drawString("This is my custom Panel!",10,20);  
      }  
 }  
 class processor {  
      private CascadeClassifier face_cascade;  
      // Create a constructor method  
      public processor(){  
           face_cascade=new CascadeClassifier("C:/USERDATA/MEDICAL/CODE/java webcam facedetect/bin/haarcascade_frontalface_alt.xml");  
           if(face_cascade.empty())  
           {  
                System.out.println("--(!)Error loading A\n");  
                 return;  
           }  
           else  
           {  
                      System.out.println("Face classifier loooaaaaaded up");  
           }  
      }  
      public Mat detect(Mat inputframe){  
           Mat mRgba=new Mat();  
           Mat mGrey=new Mat();  
           MatOfRect faces = new MatOfRect();  
           inputframe.copyTo(mRgba);  
           inputframe.copyTo(mGrey);  
           Imgproc.cvtColor( mRgba, mGrey, Imgproc.COLOR_BGR2GRAY);  
           Imgproc.equalizeHist( mGrey, mGrey );  
           face_cascade.detectMultiScale(mGrey, faces);  
           System.out.println(String.format("Detected %s faces", faces.toArray().length));  
           for(Rect rect:faces.toArray())  
           {  
                Point center= new Point(rect.x + rect.width*0.5, rect.y + rect.height*0.5 );  
                Core.ellipse( mRgba, center, new Size( rect.width*0.5, rect.height*0.5), 0, 0, 360, new Scalar( 255, 0, 255 ), 4, 8, 0 );  
           }  
           return mRgba;  
      }  
 }  
 public class window {  
      public static void main(String arg[]){  
       // Load the native library.  
       System.loadLibrary("opencv_java245");       
       String window_name = "Capture - Face detection";  
       JFrame frame = new JFrame(window_name);  
    frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);  
    frame.setSize(400,400);  
    processor my_processor=new processor();  
    My_Panel my_panel = new My_Panel();  
    frame.setContentPane(my_panel);       
    frame.setVisible(true);        
       //-- 2. Read the video stream  
        Mat webcam_image=new Mat();  
        VideoCapture capture =new VideoCapture(0);   
    if( capture.isOpened())  
           {  
            while( true )  
            {  
                 capture.read(webcam_image);  
              if( !webcam_image.empty() )  
               {   
                    frame.setSize(webcam_image.width()+40,webcam_image.height()+60);  
                    //-- 3. Apply the classifier to the captured image  
                    webcam_image=my_processor.detect(webcam_image);  
                   //-- 4. Display the image  
                    my_panel.MatToBufferedImage(webcam_image); // We could look at the error...  
                    my_panel.repaint();   
               }  
               else  
               {   
                    System.out.println(" --(!) No captured frame -- Break!");   
                    break;   
               }  
              }  
             }  
             return;  
      }  
 }  

PS.: Check out this video on the Android 3D display I build based on this stuff and the details here.
PS1: Click here to see the index of these series of posts on OpenCV
PS2: This is the older version:
 /*  
  * Captures the camera stream with OpenCV  
  * Search for the faces  
  * Display a circle around the faces  
  */  
 import java.awt.*;  
 import java.awt.image.BufferedImage;  
 import javax.swing.*;  
 import org.opencv.core.Core;  
 import org.opencv.core.Mat;  
 import org.opencv.core.MatOfRect;  
 import org.opencv.core.Point;  
 import org.opencv.core.Rect;  
 import org.opencv.core.Scalar;  
 import org.opencv.core.Size;  
 import org.opencv.highgui.VideoCapture;  
 import org.opencv.imgproc.Imgproc;  
 import org.opencv.objdetect.CascadeClassifier;  
 class My_Panel extends JPanel{  
      private static final long serialVersionUID = 1L;  
      private BufferedImage image;  
      private CascadeClassifier face_cascade;  
      // Create a constructor method  
      public My_Panel(){  
           super();   
           //String face_cascade_name = "/haarcascade_frontalface_alt.xml";  
           //String face_cascade_name = "/lbpcascade_frontalface.xml";  
           //-- 1. Load the cascades  
           //String str;  
           //str = getClass().getResource(face_cascade_name).getPath();  
           //str = str.replace("/C:","C:");  
           //face_cascade_name=str;  
           //face_cascade=new CascadeClassifier(face_cascade_name);  
           face_cascade=new CascadeClassifier("C:/USERDATA/MEDICAL/CODE/java webcam facedetect/bin/haarcascade_frontalface_alt.xml");  
           //face_cascade=new CascadeClassifier("C:/eclipse/opencv/data/haarcascades/haarcascade_frontalface_alt.xml");  
           if(face_cascade.empty())  
           {  
                System.out.println("--(!)Error loading A\n");  
                 return;  
           }  
           else  
           {  
                      System.out.println("Face classifier loooaaaaaded up");  
           }  
      }  
      private BufferedImage getimage(){  
           return image;  
      }  
      public void setimage(BufferedImage newimage){  
           image=newimage;  
           return;  
      }  
      /**  
       * Converts/writes a Mat into a BufferedImage.  
       *   
       * @param matrix Mat of type CV_8UC3 or CV_8UC1  
       * @return BufferedImage of type TYPE_3BYTE_BGR or TYPE_BYTE_GRAY  
       */  
      public BufferedImage matToBufferedImage(Mat matrix) {  
        int cols = matrix.cols();  
        int rows = matrix.rows();  
        int elemSize = (int)matrix.elemSize();  
        byte[] data = new byte[cols * rows * elemSize];  
        int type;  
        matrix.get(0, 0, data);  
        switch (matrix.channels()) {  
          case 1:  
            type = BufferedImage.TYPE_BYTE_GRAY;  
            break;  
          case 3:   
            type = BufferedImage.TYPE_3BYTE_BGR;  
            // bgr to rgb  
            byte b;  
            for(int i=0; i<data.length; i=i+3) {  
              b = data[i];  
              data[i] = data[i+2];  
              data[i+2] = b;  
            }  
            break;  
          default:  
            return null;  
        }  
        BufferedImage image2 = new BufferedImage(cols, rows, type);  
        image2.getRaster().setDataElements(0, 0, cols, rows, data);  
        return image2;  
      }  
      public void paintComponent(Graphics g){  
            BufferedImage temp=getimage();  
            g.drawImage(temp,10,10,temp.getWidth(),temp.getHeight(), this);   
      }  
      public Mat detect(Mat inputframe){  
           Mat mRgba=new Mat();  
           Mat mGrey=new Mat();  
           MatOfRect faces = new MatOfRect();  
           inputframe.copyTo(mRgba);  
           inputframe.copyTo(mGrey);  
           Imgproc.cvtColor( mRgba, mGrey, Imgproc.COLOR_BGR2GRAY);  
           Imgproc.equalizeHist( mGrey, mGrey );  
           face_cascade.detectMultiScale(mGrey, faces);  
           //face_cascade.detectMultiScale(mGrey, faces, 1.1, 2, 0|Objdetect.CASCADE_SCALE_IMAGE, new Size(30, 30), new Size(200,200) );  
           //face_cascade.detectMultiScale(mGrey, faces, 1.1, 2, 2//CV_HAAR_SCALE_IMAGE,  
           //          ,new Size(30, 30), new Size(200,200) );  
           System.out.println(String.format("Detected %s faces", faces.toArray().length));  
           for(Rect rect:faces.toArray())  
           {  
                Point center= new Point(rect.x + rect.width*0.5, rect.y + rect.height*0.5 );  
                Core.ellipse( mRgba, center, new Size( rect.width*0.5, rect.height*0.5), 0, 0, 360, new Scalar( 255, 0, 255 ), 4, 8, 0 );  
           }  
           return mRgba;  
           }  
 }  
 public class window {  
      public static void main(String arg[]){  
       // Load the native library.  
       System.loadLibrary("opencv_java245");       
       String window_name = "Capture - Face detection";  
       JFrame frame = new JFrame(window_name);  
    frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);  
    frame.setSize(400,400);  
    My_Panel my_panel = new My_Panel();  
    frame.setContentPane(my_panel);       
    frame.setVisible(true);        
       //-- 2. Read the video stream  
    BufferedImage temp;  
        Mat webcam_image=new Mat();  
        VideoCapture capture =new VideoCapture(0);   
    if( capture.isOpened())  
           {  
            while( true )  
            {  
                 capture.read(webcam_image);  
              if( !webcam_image.empty() )  
               {   
                    frame.setSize(webcam_image.width()+40,webcam_image.height()+60);  
                    //-- 3. Apply the classifier to the captured image  
                    // At this point I was wondering where this should be done.  
                    // I put it within the panel class, but maybe one could actually  
                    // create a processor object...  
                    webcam_image=my_panel.detect(webcam_image);  
                   //-- 4. Display the image  
                    temp=my_panel.matToBufferedImage(webcam_image);  
                    my_panel.setimage(temp);  
                    my_panel.repaint();   
               }  
               else  
               {   
                    System.out.println(" --(!) No captured frame -- Break!");   
                    break;   
               }  
              }  
             }  
             return;  
      }  
 }  

73 comments:

  1. Very cool; the java interface is really terrific, OpenCV did a great job with it.

    ReplyDelete
    Replies
    1. @Anonymous: 2 things:
      1. I agree with you! I really appreciate the work from the OpenCV folks and in general, from all open source code developers.
      2. You were my first ever commenter! YEIH! :)

      Delete
    2. man i dont know why but my i run the program in eclipse java and i have this error
      Exception in thread "main" java.lang.NullPointerException
      at Proyecto.My_Panel.(dale.java:32)
      at Proyecto.dale.main(dale.java:122)
      i changed the directon of face_cascade=new CascadeClassifier("C:/USERDATA/MEDICAL/CODE/java webcam facedetect/bin/haarcascade_frontalface_alt.xml"); but nothing happen still same error
      i dont know how to fix that and i am really new at this
      if u can help me i will appretiate so much cuz i dont know what to do

      Delete
    3. I think the error you are getting is not from not finding the xml file. Actually, if the file is not there or wrong address it will not tell you anything unless you purposely catch it (which the code does on the "if (face_cascade.empty())...")

      I saw this kind of error before but don't remember details... I look at http://javaeesupportpatterns.blogspot.com/2012/01/javalangnullpointerexception-how-to.html
      and it seems that some object you are trying to use is null. Did you use my code as is, on the top, or did you modify anything?

      One clue comes from looking at the line. Check for objects there that may be null. I remember being not so simple. Follow also the advices on the link (about logging info and catching exceptions). Somehow you get 2 lines with that error... Can you paste them here?

      PS.: I am not an expert on this at all. I just post things as I learn on my spare time... The person on the link above may be able to help too. But please, if you find an error on my code, let me know...

      Delete
    4. str = getClass().getResource(face_cascade_name).getPath(); (32)
      My_Panel my_panel = new My_Panel(122);
      those are the two line with the error.
      and i dont really know what you are trying to do there,

      do i need another program beside eclipse and java to open the interface webcam or does the program open that?

      i just modify the direction in this line
      face_cascade=new CascadeClassifier("C:/USERDATA/MEDICAL/CODE/java webcam facedetect/bin/haarcascade_frontalface_alt.xml");
      with
      face_cascade=new CascadeClassifier("C:/opencv/data/haarcascades/haarcascade_frontalface_alt.xml");
      also thanks for the help man i really need this to work and great post :)

      Delete
    5. One quick question... Are you the same Anonymous as the top post on July 23th? Otherwise he may be getting this thread notifications because you reply to his thread... and may be getting tired of us :)

      Just in case, I'll place the reply as a new post...

      Delete
    6. Wops, I meant as new comment... right below...

      Delete
  2. The first one is easy... Not sure why you get the error there, but you actually don't need that line. I left that and other lines from a different method to obtain the directory where the xml is, and create the path to access it. It is illustrative because there is actually some kind of bug that makes you go and have to modify the path by hand. But anyhow, in the end I just get directly the file with the same line that you are using, so, no need for all the above. So, I edited the code to comment out those. You can do the same (or copy again from above).

    The 2nd one is more tricky... I am guessing that what you actually wrote in the code is:
    My_Panel my_panel = new My_Panel();
    And not: My_Panel my_panel = new My_Panel(122); I mean, the 122 is the line... behind...

    Based on what I searched on Internet, can't really understand why it would say this is null. Got to search more but I thought I would answer you the rest first...

    Besides Eclipse and Java, to run the above you will need OpenCV. I am guessing you have this or you would have got already a ton of other errors. Did you manage to run any OpenCV code/tutorial in the past? I have couple of posts on setting that up:
    http://cell0907.blogspot.com/2013/05/opencv-quick-install-guide.html
    http://cell0907.blogspot.com/2013/06/opencv-in-c.html

    And look also for even simpler examples that the above:
    http://cell0907.blogspot.com/2013/06/creating-windows-and-capturing-webcam.html

    ReplyDelete
    Replies
    1. man thanks you so much for your help
      i am a new guy, the reason why i didn't make a new comet is because i didn't see the add comment bottom :P
      i am using opencv and i already load the library but i am not using 2.4.5 instead i am using 2.4.6
      thanks anyway for give look out for that
      now i have a new problem
      Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
      but the program runs and detect my face (weird)
      thanks for take time and answer me and help me with those link (really help me there)
      i am really new at JAVA code and this is like my first "big project" :D


      Delete
    2. So, how did you fix the NullPointerException? Was something with the setup? No need to explain if it is too long...

      And, funny enough, I was getting the same error as you "AWT-EventQueue-0" java.lang.NullPointerException

      I was ignoring it because as you said, the program worked... But then digging just now I finally found the reason! I think the paintComponent may get called before any image has been capture from the webcam, as the object was created ahead of the capture. Therefore, initially, it may try to run and have no image! The solution is to check before g.drawImage if the image is null and if so, leave without doing anything:
      if (this.image==null) return;

      On the process, I read my code and now that I know a bit more, I really don't like the way I cut the solution, so, I'll post an update of the code in a new post. I will also put a link on this, to that one, so, that other folks can find it... I thought I could just change it, but somebody may learn of the mistakes...

      Thanks for the feedback!! :)

      Delete
    3. Just to let you know that I updated this post with the new code. Help yourself ^^

      Delete
    4. so u ask me how i fix the null pointer exception?
      well doing what u said, i just erase that part of the program where was the get resource and it worked perfectly
      now i am trying to recognize eyes and mouth but i have errors :(
      thanks far updating the post so people dont make same mistakes as i did and also thanks for your time and help :D

      Delete
    5. hey man i wanna know something in your code
      what is that variable that control the face detection?

      Delete
    6. Not sure I completely follow your question...
      1. face_cascade is a CascadeClassifier object that will control and do the face detection.
      2. You initialize it by telling it what needs to look for. That is done at the "construction" time, in the line: face_cascade=new CascadeClassifier("C:/USERDATA/MED...
      3. The description of what to look for ("something that looks like a face") is in: haarcascade_frontalface_alt.xml.
      4. The detection is then done when you call the detectMultiScale method on the face_cascade object: face_cascade.detectMultiScale(mGrey, faces);
      5. mGrey is the picture where to look for the faces.
      6. The variable faces stores the results (coordinates where we got hits).

      You can check other methods of the CascadeClassifier object + the rest of the tutorial, at http://docs.opencv.org/modules/objdetect/doc/cascade_classification.html. Sorry if this is too simple or was not what you are asking... but won't hurt for other readers :). By the way, if you are not the same anonymous as the previous posts, please open a separate comment (that will avoid sending notifications to the previous commenter). And if you are, well, then nothing, but just in case :)

      Delete
    7. i am the same one :), just that i have done some modifications to your program to also detect eyes, but now i need that also detects the stage of the eyes (if they are close or open), sorry to for keep bothering u is just that i need to finish this as soon as possible, also thanks for all the tutorials and the anwser

      Delete
    8. HA HA! No! You don't bother me... Just, in return, go help somebody else :)
      Anyhow, I have not worked with other classifiers than the ones that come with OpenCV. The open eyes one is available. But I don't think there is one for close eyes (?), so, you'll have to generate it. I don't know how to generate new ones although I have seen tutorials on it.

      Delete
    9. And glad I could help... probably you already knew all this... :). Thank you for stopping by!

      Delete
  3. I just tried this example on a mac with the iSight camera and it didn't work at first. I added a delay with Thread.sleep(500) (try-catch surrounded, of course) and it instantly started working.

    ReplyDelete
    Replies
    1. Hey Dustin! Thanks a lot for the hint!! :)

      Delete
    2. This worked for me on my pc laptop too. Needed 1000ms sleep

      Delete
  4. Thanks for posting this. It is a great example of how to detect faces. Can you also provide some guidance on training the system for facial recognition? I want to take three images of each subject, find their faces and crop them out. Then I want to train the system with those three images and be able to compare future pictures of the subject to the database. I have looked through the API and am finding it difficult to recreate the C++ code in Java (and to locate certain of the C++ objects in Java).

    Great useful post. Thanks for taking the time to write it.

    ReplyDelete
    Replies
    1. Hi Howard! Thanks a lot for the complements!!! On your question, unfortunately I have never done it. I want to, but I am working on something else now so it'll be some time to get to it... Sorry!! :(

      Delete
  5. Exception in thread "main" java.lang.UnsatisfiedLinkError: no opencv_java245 in java.library.path
    at java.lang.ClassLoader.loadLibrary(Unknown Source)
    at java.lang.Runtime.loadLibrary0(Unknown Source)
    at java.lang.System.loadLibrary(Unknown Source)
    at com.test.window.main(window.java:105)



    How to get the opencv_java245.dll file

    please give me ans as soon as possible

    ReplyDelete
    Replies
    1. Will this work for you?
      http://cell0907.blogspot.com/2013/05/opencv-quick-install-guide.html

      Delete
    2. Just copy the file (opencv_java245.dll) from the path
      (opencv/build/java/x86) or (opencv/build/java/x64)
      to the path
      (opencv/build/x86) or (opencv/build/x64)
      ----
      or change the Path variable in windows settings

      Delete
  6. how can i create xml file for this program

    ReplyDelete
    Replies
    1. Are you talking about the haarcascade_frontalface_alt.xml? That comes with OpenCV. You don't need to create it. Look in your OpenCV directory:
      opencv/data/haarcascades

      Delete
    2. aaahhhh finnaly found it here.
      I think you should explain this on your tutorial.
      It took me 20 minutes just to find this lol :-D

      Delete
  7. i used ur code but its unable to open my cam i am using deafult cam or system cam of laptop\

    ReplyDelete
    Replies
    1. Sorry but in my case, it worked straight away, so, I am not sure how I would fix it... Do you get any kind of error? Does it get pass the capture.isOpened()?

      Delete
  8. Hi, My name is Gastón, I´m from Colombia. I´m using your example as a start point. It worked for me very well. Now i´m interested in change some stuff in order to shift to other focuses, for example, process a video from a cam attached to a traffic light and make a traffic control system based on the video analysis. Can you help me with some info about how to hand OpenCV. Thanks a lot and congrats !. You made a great job !. My email is gtrujillo at tecnocomfenalco dot edu dot co

    ReplyDelete
  9. Hey Gaston! Looks like a cool project :). You can do that, every time that the system sees your car, the light turns green :P.

    Anyhow, for a real traffic light, I am guessing that you got to choose an embedded system and you are probably much more familiar than me on that. :) I would look for something powerful enough to run OpenCV, but not too powerful/expensive, that can compile C/C++ and can sit outdoors. Your PC OpenCV code probably will run there after re-compile, except for the portion that reads the camera, which may change from system to system... But once you get that image in the BufferedImage, then it should work like working on a PC...

    Anyhow, I am not an expert on this, so, I am just taking a guess... It would be cool if you come back here an post your choice. On that sense, you may also want to read this and post an answer there: http://stackoverflow.com/questions/1588349/opencv-on-embedded-platform

    Good luck!!

    ReplyDelete
  10. the code runs fine without error, i get an applet screen that too blank , nothing happens and i get the following print at console :

    --(!)Error loading A

    --(!) No captured frame -- Break!

    Please help on this .

    ReplyDelete
    Replies
    1. You seem to be actually getting 2 errors, one for not getting a webcam and the other for not getting the CascadeClassifier file.

      For the first one, have you tried the code here http://cell0907.blogspot.com/2013/06/creating-windows-and-capturing-webcam.html and worked? Have you ever tried any other codes using your webcam and worked? You can try modifying slightly the code and checking the boolean output of the capture.read(webcam_image). See http://docs.opencv.org/modules/highgui/doc/reading_and_writing_images_and_video.html#videocapture-read for what it should be returning. Basically looks like a problem with your webcam...

      The second is more related to being able to read the right file... Check the path where your haarcascade...xml file is. Try reading what that path really is (you may have a surprise... I did...).

      Good luck!

      Delete
  11. Hi,

    I haven't read through all comments, so this might be already in here. I've just looked through your conversion code and going through jpg would result in quality loss and a lot of computation overhead. I've just had the same problem (google sent me here) and found a more efficient way to do it. It's obviously dependent on hardware, but a 720p color image conversion takes about ~1.4ms on an i7 ultrabook.

    code:
    private static BufferedImage cvMat2BufferedImage(final Mat matBGR){
    // read data from mat
    int width = matBGR.width(), height = matBGR.height(), channels = matBGR.channels() ;
    byte[] sourcePixels = new byte[width * height * channels];
    matBGR.get(0, 0, sourcePixels);

    // create new image and get reference to backing data
    BufferedImage img = new BufferedImage(width, height, BufferedImage.TYPE_3BYTE_BGR);
    final byte[] targetPixels = ((DataBufferByte) img.getRaster().getDataBuffer()).getData();

    // copy data
    System.arraycopy(sourcePixels, 0, targetPixels, 0, sourcePixels.length);
    return img;
    }

    ReplyDelete
    Replies
    1. Man! Thanks a lot for your input. Sorry it took me so long to be back to you, but I needed enough spare time to try this out. Basically, there has been different comments about doing it this or that other way. Never looked at the efficiency, but finally, triggered by your comment, I went back and check. You can see the full test here: http://cell0907.blogspot.com.es/2013/12/from-mat-to-bufferedimage.html

      Basically, your method is the best of all. It was pretty close to another one I had found but still better. You save on the R/B channels flip, and honestly, not sure how you do it. I can see it but I don't think I fully follow. Anyhow, I will update the code above with your method. It is big savings!
      Thanks again!!

      Delete
  12. Hi Cell0907:
    As i said earlier i am working on my final term project , so i am gona ask you that can i connect multiple cameras and process their streamings? Also whether i will be able to connect to a 4-Channel DVR and process the streaming for each cam separate from one another( At same Time), or i should to choose IP cameras for that , ohhh... i have asked too much, so let me give some brief detail of my project that will help in Understanding my Questions:
    I want to develop a System that will be connected to Multiple cameras (1-2-3 or 4) , the data coming from the cameras will be processed and faces will be detected and recognized and will be stored in database.
    So i will be thankful to you for your response and suggestions , also as too many peoples are demanding , that will be good if you can post regarding FACE RECOGNITION.

    ReplyDelete
    Replies
    1. Hi Adnan,

      Sorry but I have to say that I have no experience with what you are asking. I have only used streams from one camera at a time, from either my laptop or cell phone camera. I appreciate the description though. I did quick check and saw this:
      http://stackoverflow.com/questions/11737606/capturing-pictures-from-multiple-webcams-4-ones-using-opencv
      http://stackoverflow.com/questions/13925867/displaying-multiple-camera-feeds-simultaneously-with-opencv
      http://stackoverflow.com/questions/7437472/multiple-cameras-with-opencv
      http://stackoverflow.com/questions/13049777/after-reading-multiple-frames-from-a-camera-opencv-suddenly-always-fails-to-rea

      They point to several problems you may encounter, but I don't think they give you the full answer.
      Still cool topic. If you ever post the answer somewhere, let me know and I'll put a link to it :)

      (FYI, I am learning on this as well... I just post whatever I learn for everybody's else benefit... so, I am far from being an expert on OpenCV...)

      Delete
  13. Cool Brother, ok there is my Last paper on 23 December, will Study that case after that and will Share My Knowledge with you:)

    ReplyDelete
  14. Hi Cell0907, as you know detection with haarcascade_frontalface_alt is Good , but speed is slow, and with lbpcascade_frontalface.xml is Fast but not so good, do you noticed that ? If yes then is there any good way to achieve both parameters, i mean fast and Good?
    If you have found any way for that ,kindly share that :)

    ReplyDelete
    Replies
    1. Hi Adnan, yes you are right! I did some benchmarking few days ago. It wasn't great because I was including other stuff, which kind of distorted things, but lbp was easily 4x faster on my cell phone (HTC One). To pick up speed (I am sure you know), the best is to reduce the image resolution. Difference can be huge (10x) as many times one has way more resolution than needed...

      Performance wise, I did not see much degradation with any of the changes, if any, although I didn't really have them all running in parallel to compare. That would be nice exercise :). Actually, when reducing the resolution I felt I reduced the number of false positives too. Performance was more solid than with better resolutions...

      Still, for the cell phone (I know, may not be your case), I think that using the embedded face detection methods in Android give the best results for similar effort, at least without becoming a master of OpenCV. They work with face rotation in any angle, where CascadeClassifier fails, and with no false positives (that I have seen).

      Ultimately, to make it work robustly and fast, with those classifiers as they are, you may have to add other constraints. For instance, filter false positives by eliminating detections that just appear instantly (and maybe disappear as fast) somewhere away from where you are having all the positives. I was just doing that in a case where I knew there was one face in the image that I needed to track... I can think of other tricks of this style, but no magic or anything advanced... Have not researched this extensively.

      Anyhow, good luck and Happy New Year! :)

      Delete
  15. Thanks for your code when i am running this code..the code runs fine without error, i get an applet screen that too blank , nothing happens and i get the following print at console : even my webcam also initializing..

    --(!) No captured frame -- Break!

    Please help on this .

    ReplyDelete
    Replies
    1. As you may be guessing, it basically looks like you are not capturing a frame from the webcam. And as you said, if you get thtat message, it pass the capture.isOpened() check. What does the read method return? http://docs.opencv.org/java/org/opencv/highgui/VideoCapture.html#read%28org.opencv.core.Mat%29

      Sorry, but have not seen this before...

      Delete
    2. sir i need a coding of detecting face for many images in a folder in java code using opencv for windows.i'm not using the webcam.

      Delete
    3. Hi Siva, will answer you on a separate comment (just below), so that Rohit doesn't get all our reply notifications...

      Delete
  16. @Siva: I guess that a part of what you are asking (detecting the face in a file) is part of the OpenCV installation original tutorial. I talked about that here: http://cell0907.blogspot.com/2013/05/opencv-quick-install-guide.html For the rest, you just need to apply the above file by file in the directory. I quickly found this http://stackoverflow.com/questions/5694385/getting-the-filenames-of-all-files-in-a-folder
    Good luck!

    ReplyDelete
  17. This one help me a lot. have you try for the face recognition. because am stacking there . if possible i need your help. What i need is detect the face as above and take the rounded face as a template so as to able compare with that in the database. I need you help. Thanks.

    ReplyDelete
    Replies
    1. Thanks for the comment, but sorry, if I understand right, I can't really help you. Howard Roark above in these comments seems that wanted to do the same (face recognition...) but I have never done it myself...

      Delete
  18. Hi there! First I want to thank you for sharing your experiences with us.
    I want to detect faces from webcam using OpenCV 2.4.9 and your post is my principal guide. So, I have changed 2 lines:
    1. In the constructor of processor class: face_cascade=new CascadeClassifier("C:/Users/Badescu/workspace/DetectieFaciala/src/detectie_de_la_webcam/haarcascade_frontalface_alt.xml");
    2. In main: System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
    Then I ran the program and there appeared a grey panel named "Capture- Face detection", the LED showing that the webcam is active lit and have 2 messages on the console:
    Face classifier loooaaaaaded up
    --(!) No captured frame -- Break!
    I've also tried to the program which aims to display the webcam from your post:http://cell0907.blogspot.ro/2013/06/creating-windows-and-capturing-webcam.html and there I got the errors that I'll describe at that post.
    Can u, please, help me to debug these errors?

    ReplyDelete
  19. Hello, I have set the path for opencv_java249.dll in my case but still it is raising an error as

    Exception in thread "main" java.lang.UnsatisfiedLinkError: no opencv_java249 in
    java.library.path
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1860)
    at java.lang.Runtime.loadLibrary0(Runtime.java:845)
    at java.lang.System.loadLibrary(System.java:1084)
    at window.main(window.java:84)

    what is the possible solution for this?

    ReplyDelete
    Replies
    1. Can't remember all the details about the install... but did you follow this?
      http://cell0907.blogspot.com/2013/05/opencv-quick-install-guide.html

      Delete
  20. Respected Sir , I'm Pravin Fatak(Student). Its really awesome your code and its work . I want code for or ideas about detect image and fetch information of person on database.(JAVA)

    ReplyDelete
    Replies
    1. I guess you are looking for coding example/ideas on how to store the faces in a database? Something that later you can take a new face and cross it against that database to find a match? Sorry, but out of luck here. Have never done it. In all my ignorance, I guess that one face alone will not work. I.e., probably you have to store the classifier, obtained from running many views of the same face? Anyhow, sorry, but just guessing here...

      Delete
    2. Hello Pravin You first learn the feature extraction teachniques from image.
      You can compare two images on the basis of images features.

      Delete
  21. Thank u alot for this version in java .. I was looking for it .
    I'm new to openCV , this is the first time i deal with this & I'm just facing some troubles in running the code correctly ..
    first i had changed the version to opencv-249 ..
    but i don' know what to write exactly instead of this path "C:/USERDATA/MEDICAL/CODE/java webcam facedetect/bin/haarcascade_frontalface_alt.xml"
    I don' understand what it really refers to !
    would you give a -general- example , plz? or tell me what i should write exactly ?
    Also , I don' know if i should put the .xml file in the resources with the code or not :-?

    because all i get when i run the code is --(!)Error loading A
    and it keeps running and saying "Detected 0 faces" , although the camera is initialized and i can see my photo ..
    and here's another thing .. how could the camera work although i'm putting the same path as yours O.o
    it shouldn't right ?

    I just need more clarification , plz
    and many thanks in advance :))

    ReplyDelete
    Replies
    1. Hi jeh,

      am facing same issue. did you fix it ?
      thanks

      Delete
  22. cell0907, big thanks to you :)
    We are in search of something for face detection and (maybe) recognition...
    This example works well. Just a right thing to take a fast look at OpenCV capabilities.
    -=-=-
    Will search now for right cascades use. :)

    ReplyDelete
  23. Sir. where do I get this System.loadLibrary("opencv_java245"); , because when I run this code, in that code has an error. What is the possible problem? Thank you..

    ReplyDelete
    Replies
    1. Read the previous comments. System.loadLibrary(Core.NATIVE_LIBRARY_NAME);

      Delete
  24. It is not working if I put it in a button.
    How can I fix this problem

    ReplyDelete
  25. i use opencv_java300. when i compile window.java, there is no error.But when i run , i found error as

    Exception in thread "main" java.lang.UnsatisfiedLinkError:org.opencv.objdetect.CascadeClassifier.CascadeClassifier_1J
    at org.opencv.objdetect.CascadeClassifier.CascadeClassifier_1
    at org.opencv.objdetect.CascadeClassifier.
    at processor.
    at window.main

    what should i do, plz?

    line 57 ==> face_cascade=new CascadeClassifier("C:/USERDATA/MEDICAL/CODE/java webcam facedetect/bin/haarcascade_frontalface_alt.xml");
    line 58 ==> if(face_cascade.empty())
    line 94 ==> processor my_processor=new processor();

    ReplyDelete
    Replies
    1. it's very useful topic :) good job bro :) but i really need to answer the question of this reply :) thx in advance

      Delete
  26. i use opencv_java300. when i compile window.java, there is no error.But when i run , i found error as

    Exception in thread "main" java.lang.UnsatisfiedLinkError:org.opencv.objdetect.CascadeClassifier.CascadeClassifier_1(Ljava/lang/String;)J
    at org.opencv.objdetect.CascadeClassifier.CascadeClassifier_1(Native Method)
    at org.opencv.objdetect.CascadeClassifier.(init)(CascadeClassifier.java:58)
    at processor.(init)(window.java:57)
    at window.main(window.java:94)

    what should i do, plz?

    line 57 ==> face_cascade=new CascadeClassifier("C:/USERDATA/MEDICAL/CODE/java webcam facedetect/bin/haarcascade_frontalface_alt.xml");
    line 58 ==> if(face_cascade.empty())
    line 94 ==> processor my_processor=new processor();

    ReplyDelete
  27. I getting nothing except that frame and a mesage classifiers successfully

    ReplyDelete
  28. The method ellipse(Mat, Point, Size, int, int, int, Scalar, int, int, int) is undefined for the type Core

    ReplyDelete
  29. hello sir i want to know how to import our system camra with my program

    ReplyDelete
    Replies
    1. Just use VideoCapture class like:
      VideoCapture capture =new VideoCapture(0);
      the (0) here means you have assigned the first camera device in your system to capture variable
      if you have multiple cameras like a usb webcam, then please select it's number

      Delete
  30. this form is realy very helpful every one top get tips and idea for shorting their problem i would like to say thanx..

    ReplyDelete
  31. when i run the code it doent give any error but no program starts it just says no errors found and succesful build. what should do i do?

    ReplyDelete
  32. I got this thing
    Face classifier loooaaaaaded up
    --(!) No captured frame -- Break!

    Why frame is not captured

    ReplyDelete
  33. Hi cell0907,

    I run your code (Captures the camera stream with OpenCV) successfully but its not detecting face, is am something doing wrong or need some additional fixes. Thanks lot for your help :)

    ReplyDelete
    Replies
    1. Hello cell0907,

      Fixed, that was xml path issue. thanks

      Delete