Over the past couple months, I've had a lot of traffic (well a lot for me anyway) going toward my various image editing posts. While the general concepts are there, I get a decent number of people complaining that they're a bit slow. They are slow when the image gets to be rather large. The reason for this is because I intentionally use two functions that are incredibly slow, GetPixel and SetPixel... These functions have to figure out the format that the image is actually displayed in, and give us/set the individual pixel in the image. To say the least that's a slow process (safe but slow). So why would I use them if they're so slow? Because they're easy to understand. Most of my code was designed such that someone who didn't know much about images or maybe even that much about C# could see what was going on and get something together for their needs. The alternative gets much more complicated, much faster...

That alternative is by using unsafe code. I can honestly say that there are only a couple times that I've ever seen someone use unsafe code (usually when doing image manipulation). So if you're a C# developer, there is a good chance that you've never (or at least rarely) seen the keyword used. So what the heck is the unsafe keyword? The unsafe keyword is used to define a section of code that uses pointers.  You can define it on a function, encapsulate a small bit of code with it, etc. You're just saying this bit of code uses pointers. If you've done C++ programming, you've dealt with pointers enough that this should be rather simple. For everyone else, well I'm not going to explain pointers that much other than to say that they point to things in memory.

OK, well maybe a little bit more is needed. In our case, think of our image as an array of bytes in memory. When we create a pointer and point it to our image, it does not hold our image. The pointer is simply an int (actually it depends on the language/architecture as to what the type is) that says we can find this image at location X in memory. On top of this we can change where the pointer points. For instance if we add 1 to it, we get the next byte in memory (1 pixel in on our image, assuming 1 pixel=1 byte). It's referred to as pointer arithmetic. That's all you need to know for our purposes, but If you want to know more Google is your friend.

Getting back to the unsafe keyword, the advantages of using pointers and unsafe code is it gets rid of a lot of bounds checks, etc. that slow down your code. The downside is that it can introduce potential security flaws if it's not written correctly... Not to mention they tend to increase the number of bugs that come about (especially with people who have limited experience with them). That being said, we're going to use unsafe code to speed up our image manipulation.

The two functions that were slowing us down were GetPixel and SetPixel. So let's create our own. So how do we do this? Well with images there are a couple steps that you need to do to get at the data so that we can set/get a pixel. The first step (after loading the image) is to lock it. I've written a function to do this step for me:

With this function we're telling the image to lock all of it (since the rectangle is the entire image) for both reading and writing. The last item is the pixel format of the image. In our case we're using the image's own pixel format, because we don't exactly know what the format is suppose to be. That being said, in later code we're going to limit ourselves to 24 and 32 bit images (expanding this for 48 or 64 bit is actually fairly simple but not something that I needed). In return the function returns a BitmapData class. This class contains our actual bits of data, but we still aren't quite ready to edit it yet. The next step is we need to figure out the actual size of each pixel (luckily for us, they tell us):

As I said earlier, we're only really going to handle 24 and 32 bit images. Once we have our data and know our pixel size, we can actually get/set our needed pixel:

That code gets/sets our individual pixels respectively. In it you'll notice that we finally use the unsafe keyword. Up until now we didn't use any pointers, but here we're using them to point to the actual data. The BitmapData class contains an IntPtr called Scan0. This is the actual data of our image... Well the pointer to our actual data. The Stride property tells us the actual width of the image. You see, if we have an image that is 207 pixels wide, it might not actually be 207 pixels wide. Images, to make them faster to manipulate and load, tend to have actual widths that are multiple of 4s (although with 64 bit processors, I wonder if it would be a multiple of 8...). So in reality the image would request enough memory such that the width was 208. The extra pixel would just be sitting out there, not doing anything. And in normal managed code, it wouldn't matter, but in unsafe pointer land we need to know about it. So using some pointer arithmetic (y*ActualWidthInBytes+x*PixelSizeInBytes), we can get the actual location of the pixel that we want. From here, you'll notice that we're accessing it in reverse order (BGR instead of RGB). I don't know why this is done in memory, but I do remember that bitmaps are actually stored in reverse order. The bottom right pixel is the first one that you come to, stored in BGR order, and you end with the upper left (no idea why this is the case though). Anyway, we get our R, G, B, and potentially A values and create a Color object and return it.

The last thing that we have to do, once we're done manipulating our image, is unlock the Bitmap object. Once again I've built a function to help with that:

That's all there is to it really. Once unlocked, we're done. We can simply replace our functions from before with these and we end up with a much faster system. For instance we can do a median filter like this:

The Math.MathHelper.Median function simply sorts the list and gets the middle item. But it's not that different from the code before (although a bit cleaned up). The speed difference though is rather amazing. For a small 274x350 image we go from 8 seconds to about 2 seconds. And that's with a simple update. If you wanted, you could increase this further by breaking out the GetPixel and SetPixel functions and doing the pointer arithmetic in the function itself (doing 1 add and 1 assignment instead of 2 multiplications, 2 adds, and an assignment and Color object creation makes things run a bit faster). But still, for a simple update that's a big speed increase.

So try it out. Copy it into your IDE and compile it (you may want to set up a function call, load up an image, save it, etc. so it's at least a somewhat interesting test), I'll wait... Oh, it failed didn't it? The reason for this is because you must compile unsafe code using /unsafe (just go into the properties of the project, build tab, and check the Allow unsafe code box). OK, now try compiling it and you should see a nice, much faster, median filter applied to your image. So try it out, leave feedback, and happy coding.

   

This one is because of a question I had someone ask me and I thought it might help out others so I figured I'd talk about the subject. Basically I've done quite a few different posts on various aspects of image manipulation. In fact I may focus a bit too much on the subject from time to time, but it's something that I find interesting. Anyway, while I've shown some of the more interesting aspects of the GDI+ code, I've never really talked about the subject of pixel formats or even image formats and how they're handled. Mainly because it's a fairly boring subject but I might as well cover it.

In GDI+, we generally use either the Bitmap or Image classes. When creating a Bitmap object, we can set the pixel format for the image (one of the constructors excepts it along with the height/width). There are a few of these but they're all fairly similar. For example, there is a format called Format16bppArgb1555. This format, is a 16-bit format that reserves one bit for the alpha channel and 5 bits each for the green, red, and blue channels. The listing of the various formats that are available can be found here. You may want to note that there is a format for 48-bit and 64-bit images. There is an issue though as it converts it down to a 24/32-bit image when you save it again. Other than this exception, once you've set the pixel format for that object, you're stuck with that format. The only way to convert it is to create a new image and copy the data over.

Image formats on the other hand are quite simple to deal with and converting from one to the other is easy. Generally when loading an image, it doesn't matter what the file format is (assuming of course that it's supported). With saving a file though, you may want it to be in a specific file format. For instance, since most of what I do at work is web related, I save a lot of things as JPEGs or PNGs. Now if I load up a Bitmap object with an image and then just try to save it as a JPEG, I'll run into an issue (anything that I load it in will say that it's not a jpeg, it may still load but it will throw up that warning). The reason for the issue is that it didn't save it as a JPEG. Instead the image is simply a bitmap file. The reason for that is that I never told it what format to save the image as and it simply used its default which is a bitmap. In order to fix this the Save function of the Bitmap class can take in a second object of type ImageFormat. This, as the name suggests, determines the format of the image. This is however not an enumeration, but a class. Thankfully there are some predefined definitions for us to use that can be found here. They're static members so you don't have to create anything yourself. Although if the format you want isn't listed, you're going to have to create your own definition (basically your own file loader). It's not a fun process...

Anyway, that's it. With that information you should be able to get the file to the specific format that you want. Sorry that there isn't any code on this one, but hopefully it helps someone out a bit. Anyway, give it a try, leave feedback, and happy coding.

   

Before I show the code, I just want you to know that this is not efficient at all. That being said, this is really just to show a concept. Anyway, I've shown a couple of times how procedural image creation can be used for some interesting effects but I've never shown what can be done with an existing image. In this instance I'm going to use a cellular map on a pre existing image. So lets look at the code for the cellular map first:

The cellular map above is really just a set of points and a list of distances. It doesn't do much really but what we've end up doing with this code is grouping sets of pixels together around a point. And what you'll notice, if you have a large number of them, is that it ends up creating some fairly interesting looking geometry. So, just for fun, I took a pre existing image and laid the cellular map over it. Then I took the groups of pixels around each point in the cellular map and averaged them together to get an "average" pixel and put the result into a new image (as you can see below):

The results end up looking like this:

And if you can't tell, that's the Mona Lisa. Like I said, it isn't a practical bit of code but it does result in interesting images. And if you want it to look even more like an oil painting, use the median instead of the average. So take a look, leave feedback, and happy coding.