Friday, November 8, 2013

OpenCV: How to get started and index of posts

So, you want to do some cool image processing... so, here is a very basic explanation on how to get started with link/index to the stuff I posted so far.

Where to start

Basically OpenCV is a set of nice libraries that will highly expedite a lot of your work. See more detail here. You will need to call them from your program, say in C/C++ or Java.

To develop the program you will need an IDE (integrated development environment) which takes care of editing (more than a text editor, with nice support features), and allows you to "plug in", compilers, debuggers, etc... so that you can run everything from there. In my case, I am using Eclipse. Notice that you could use a different IDE. During configuration of the IDE, you tell it what compiler to use, etc... So that when you press the compile button, Eclipse calls it to generate the object file... I think you get the point...

So, install Eclipse if you don't have it. Of course, you could choose a different IDE, but sorry, no experience on that, and honestly, I have no complains about Eclipse. Great tool. In my case, I did the installation when I was planning to use it with Qt (a set of widgets for something that I was doing completely unrelated). Still, the initial portion of the installation is the same. You can see my installation experience here.

So, what do I choose, C/C++ or Java?

Not an expert on this but I believe that a lot of the initial stuff on OpenCV was done for C development. Nevertheless, I think that almost everything in OpenCV has a port to Java already. I.e., there are packages in Java equivalent to libraries in C. So, probably depends more on what language you feel comfortable with.

One thing I was thinking (and I was wrong) is that if I want to use OpenCV in Android, I better get a handle of the Java version, as that is the language to develop in that platform (see this overview on Android programming environment). Nevertheless, that it's not that truth. In Android development, there is something called the NDK which allows you to call pieces of C code (native code) from the java code (see more info here on the NDK). The funny part is that in the end, as OpenCV is pretty processing intensive, you may want to use the NDK for the OpenCV routines (native code runs much faster than java code, if you do things properly...). Bottom line, I went down the path of working with OpenCV in Java, but probably I (and you, if you prefer) could have saved part of that work, as I was more familiar with C. But as I went down that path, I end-up with Eclipse supporting both languages (that is done with the "views"). So, to install each of the languages to work with OpenCV in a PC CPU, follow:
  1. Installation to work with C/C++
  2. Installation to work with Java
To develop for an Android machine, follow this.

Note: The 3 links above are in the order of what I historically did on my case. I.e., the 3 steps may not be completely independent of each other. For instance, something got installed in step 2 that is needed for Android, but I never realized and if I had done only the third, it would not have worked. Please let me know if you got any trouble...

Coding examples

So, now that you got it installed, let's do some coding:
  1. As part of the installation of OpenCV in Java (the same link as above) we did a first tutorial where we detect faces in an image file in a PC.
  2. Creating windows in Java and drawing in them. We strip everything to get familiar with the java drawing/workings. There is no OpenCV here. I read an image file from disk and display it. No detection yet.
  3. Next, we create windows and capture webcam, i.e., not from a file, like in #1. Although we do not do any image processing, we do use OpenCV structures to be ready for the next step (processing). 
  4. One of the keys on that post is how to take a Mat structure from OpenCV and pass it to BufferedImage, for display in Java. I created a post just for that.
  5. And into image processing, we now detect the faces on the webcam stream. Notice that this is very similar (a port to Java) of the original C tutorial
  6. Then we move into detecting a ball in the image. Part of the detection is based on color. So, you got to understand the color space. See this.
  7. And then we track the ball with the PC camera/CPU.
  8. Then we start moving into Android with a first app using OpenCV.
  9. Here we track faces with OpenCV in Android, using the smart phone camera. And here I use the Android SDK for the same thing.
  10. Using the face tracking I created a 3D display, which seems to be a similar technique to what Amazon is going to be using on their phone.
  11. And finally, we port the tracking of the ball to Android, i.e., we use the smart phone camera/CPU.
  12. The last final thing that I had in plan but haven't done yet is to actually port it to Android but have the OpenCV CPU intensive routines done in a C library, with the NDK.
For the next posts I am going to be concentrating in Android development, not related to OpenCV, but I'll be back :)

Cheers!!

14 comments:

  1. Just stumbled across this blog (I'm working on Motion Detection in OpenCV Java) and this is like a treasure trove of much needed help!

    Thanks, and keep up the good work!

    -- Will

    ReplyDelete
  2. Thank you so much.....nice work.. keep it up...

    ReplyDelete
  3. Amazing, keep it up :)

    ReplyDelete
  4. Thank you for your Tutorials on Java OpenCV , We are using it at the university,you are like the only ohne who makes OpenCV Tuts for Java. Thank You, please make more.

    ReplyDelete
  5. I don't why I get this error: "The method ellipse(Mat, Point, Size, int, int, int, Scalar, int, int, int) is undefined for the type Core "
    public Mat detect(Mat inputframe){
    Mat mRgba=new Mat();
    Mat mGrey=new Mat();
    MatOfRect faces = new MatOfRect();
    inputframe.copyTo(mRgba);
    inputframe.copyTo(mGrey);
    Imgproc.cvtColor( mRgba, mGrey, Imgproc.COLOR_BGR2GRAY);
    Imgproc.equalizeHist( mGrey, mGrey );
    face_cascade.detectMultiScale(mGrey, faces);
    System.out.println(String.format("Fete detectate: %s", faces.toArray().length));
    for(Rect rect:faces.toArray())
    {
    Point center= new Point(rect.x + rect.width*0.5, rect.y + rect.height*0.5 );
    Core.ellipse( mRgba, center, new Size( rect.width*0.5, rect.height*0.5), 0, 0, 360, new Scalar( 255, 0, 255 ), 4, 8, 0 );
    }
    return mRgba;
    }

    ReplyDelete
    Replies
    1. You could use Imgproc.ellipse with OpenCV 3.0

      Delete
  6. I am the above "Anonymous". I solved the problem by using OpenCV2.4.9 instead of 3.0

    ReplyDelete
    Replies
    1. Hi anonymous, wish I had time to stop by and help folks (working on something completely different lately), but thank you for coming back and posting the answer to your question. Shows nice etiquette. Cheers!

      Delete
  7. man this website is awesome. i search for a month. finally i got what i need. thanks man

    ReplyDelete