bensmiley's blog

MIDI Driven Animation using CoreMIDI in Objective C

So in this post I'm going to explain how to produce MIDI driven animation on OSX or iOS using the CoreMIDI and CoreAudio frameworks. When I first started trying to do this I thought it would be easy - just register a callback in the MIDI player which is called every time a MIDI message is played. Unfortunately this is not possible and I ended up spending three long days figuring it out from the limited documentation available. Hopefully this post will save someone some time!

Project files

A fully working X-Code project can be downloaded here.

The Goal

In this guide I will explain how to do the following:

  • Load and play a MIDI sequence from a file using a MusicPlayer
  • Play the MIDI notes with an instrument effect (SoundFont) using an AUGraph
  • Create a virtual endpoint to intercept and display the MIDI messages in realtime

Load and play a MIDI Sequence

The following tasks are needed to load and play a MIDI file:

  • Create a MusicSequence to hold the MIDI information
  • Get a NSURL to hold the path to the MIDI file
  • Load the sequence file into the sequence using MusicSequenceFileLoad
  • Create a new MusicPlayer, add the sequence and play the sequence

Now here's the code. You will need to include the following frameworks: CoreAudio, CoreMIDI and AudioToolbox as well as the import: AudioToolbox/MusicPlayer.h


Parsing bitwise operations from a string in Java for Box2D

This is a short post which might be helpful to some people who are using Box2D. If you want to filter collisions in Box2D you need to use mask bits. In order to manipulate mask bits you need to use Java bitwise operations. For a project I'm working on I need to store these mask bits and bitwise operations in a text file. The information then needs to be parsed by Java to create my mask bit. I need to parse strings in the form: ~0xFFFF&0x0001. Java doesn't seem to be able to parse this kind of string natively so I wrote a simple function which does the job:


Connecting objects to Sprites in Box2D

This topic seems to generate a lot of confusion when people first start using Box2D. Here's a simple explanation. Box2D has nothing to do with display images. Box2D is just a physics engine which means it tracks the position and velocity of the bodies it contains. To display the bodies you need another system. This could be DirectX, OpenGL, JavaFX or any other graphics system.

Once you've got your graphics up and running and you can display images on the screen the question becomes "how do you connect the position and rotation of the sprite with the position and rotation of the body contained within Box2D?".

To solve this problem I wrote a simple connector class which can be used to link a sprite to its corresponding body.


Tracing images to create Box2D objects - part 2

Now that we have our outline we need to find a way of reducing the number of points whilst keeping the path accurate to the outline of the shape.

To do this we want the number of points to be dependent on the curvature of the path. Points of greater curvature need to have more points and sections which are "straighter" need to have fewer points. This can be done by measuring the curvature of the line - when the line has a greater curvature we add more points. One difficulty is that because the line follows every pixel it's has lots of small duration high curvature sections. We need some way to average the curvature. As yet I've not found a perfect solution but I've written an algorithm which seems to do a job which is good enough.

Algorithm 1

Average the points

To get rid of local areas high curvatures we need to average the points. The average can be found by adding the coordinates of a number of points and then dividing by the number of points.

The slight problem with this is that the new line will diverge slightly from the old line due to the nature of averaging. The averaging function just loops over the points and computes a new average point based on a number of points.


Tracing images to create Box2D objects - part 1

Today I was looking at the forums and I noticed that there was a level creator for Box2D which had a feature which allowed image files to be converted onto Box2D objects. This program traced around the edge of the image and then output a series of points which could be used to create the Box2D shape. I've written a series of tutorials on how to create Box2D objects using Inkscape SVG images but when you have a sprite and just want to convert it straight to Box2D this technique looks easier. I decided to see if I could write my own script to do this.

My first thought was that it made sense to use the image's alpha channel to determine the outline. Then it was necessary to scan the pixels in the image and determine which ones constituted the outer edge. Finally it was necessary to reduce the number of edge points (for even a fairly small image there could be over 1000 points) whilst not losing the characteristic of the shape.

Finding a boundary point

Finding which points are on the outline of the shape is fairly easy. Any boundary point will have an alpha value greater than zero and at least one neighbor with an alpha value of zero. Here's my "find boundary point" function:


Screen to World coordinates in Box2D

In this blog entry I'm going to talk about how to manage the coordinates system in your Box2D world. The problem arises because normally in games we want to use two sets of coordinates - one for the Box2D world and one for the screen output. Time and again I come across people using sub-optimal methods for managing the two sets of coordinates.

Mistake 1: Pixel coordinates

When people start out with Box2D normally they create a Box2D world with the same dimensions as the screen on which their world will be displayed i.e. pixel dimensions. If the game runs at 1000px x 800px then the Box2D world has dimensions 1000m x 800m. This is a simple and easy solution but can cause all sorts of problems.

Firstly, Box2D is optimised to run using real world dimensions i.e. objects ranging in size from 0.1m to 10m. Having 100m meter objects flying around will reduce performance and could lead to bugs and glitches.

Secondly, this method is horribly inflexible. What if you want to modify your game to run at 500px x 400px? The short answer is that you can't!


The second method I see is to use a global parameter SCREEN_TO_WORLD which contains the ratio of Box2D units to screen pixels. This method allows you to run your Box2D world with sensible dimensions but still doesn't offer much flexibility.

What if you want to zoom in on a section of your world? What if you want to pan across your world like in Angry Birds? What if you want the coordinates from your world to go from -50 to +50 but your screen needs to go from 0 to +500? The answer to all these questions is simply "you can't!".

This method is also hacky! Your code ends up littered with references to SCREEN_TO_WORLD and WORLD_TO_SCREEN which isn't object orientated or flexible. It makes the code hard to read which in turn makes it much easier to make mistakes.


Creating levels in Box2D Part 3 - Extracting Inkscape path data using Batik

Welcome to the third part of my tutorial series on how to use Inkscape to create objects for Box2D physics simulations. In the first part I described how to load the SVG file into Java and start extracting the information we need. In the second section how to interpolate cubic splines. In this section I'm going to describe how to parse the SVG path data using the open source library Batik.

Firstly you need to download Batik here and make the library available in your software development environment. I use Eclipse for my Java development and add external libraries as user libraries. For instructions on how to do this visit here.

Once Batik is set up we're going to start by extracting the transform data from the SVG document.

Parsing transform data

To parse the transform we're going to use the Batik TransformListParser class with the TransformListHandler interface. Batik uses an event based parsing model. This means that as parser parses the code for each command it finds it calls relevant method in the handler i.e. matrix command calls matrix() method. Batik provides an implementation of the handler interface for AWT but we want to write out own implementation. To do we need to create a new class which implements the TransformHandler interface. Then in each of the methods we can decide what we want to do when this event occurs.


Creating levels in Box2D part 1 - Parsing Inkscape SVG files in Java

Welcome to the first in a series of guides on how to create Box2D levels using by loading SVG images from Inkscape.

In this series I'm going to cover the whole process: loading the load the SVG document into the java enviroment, to parsing the SVG data using the open source library Batik to extract object, generating Benzier curves different levels of accuracy, triangulating your shapes using a constrained Delaunay triangulation and finally creating the Box2D objects and adding them to your Box2D world.

Firstly, what's the motivation for doing this? Designing levels for games manually is time consuming. Firstly you need to design the level then work out the coordinates for each point. Because Box2D can only work with convex objects with a maximum of 8 sides you need to decompose your level into convex objects.

The motivation is simple, it's much easier to design your levels in a paint program and then load them into Box2D than to manually enter the coordinates. You can buy software to do this for you but it's much better to know how to do it yourself than using a black box solution.

One word of warning. To follow this guide you will need to be proficient using Java and understand the basics of XML and the Document Object model.

Part 1: Loading your SVG document into Java

For this tutorial I'll be using Inkscape which is available to download for free here. The first step in the process is loading your SVG document into Java after that it will be possible to extract the path and shape data. You could load the file as a string but then you would have to write code to parse it to extract the path data. A better solution is to use pre-existing functionality to load the file as an XML document. After that you will be able to traverse the DOM to extract the relevant tags.

I tried using the XML loading functionality included in Batik but it was very slow - usually taking 3 - 4 seconds to load a simple SVG file. In the end I found it was better to use the standard Java XML parsing functionality. Below is the function I used:


Creating levels in Box2D part 2 - Creating Bezier Curves in Java

In this post I'll continue my series on how create objects for Box2D using an SVG file created in a drawing program like Inkscape. In Part 1 I covered how to load the SVG file into Java. Before I explain how to extract the path data from the SVG file it's necessary to know how we are going to use this data to construct our paths. SVG paths can contain a number of different types of line: straight, quadratic bezier, cubic bezier and arc. In this tutorial I'm going to cover straight, quadratic bezier and cubic bezier lines.

A bezier curve is a curve defined by a cubic polynomial. The maths is pretty complicated but luckily it's quite easy to construct bezier curves geometrically.

Cubic bezier

A cubic bezier is a curve defined by it's start point P0, it's end point P3 and two control points. The control points P1, P2 define the gradient at the start and end of the line.

Quadratic bezier

A quadratic bezier is just a cubic bezier curve where control points P1 and P2 inhabit the same location.

Straight line

A straight line is just a cubic bezier curve where the control points P1 and P2 lie on the start and end points P0 and P3.

Constructing the line

From this analysis it can be seen that if we can draw a cubic bezier curve we can by definition use the same code to draw quadratic beziers and straight lines.

The diagram above shows how a cubic bezier curve can be constructed geometrically. We start out with a curve defined by two points and two control points P0 with control point P1 and P3 with control point (cp) P2. By performing a number of bisections we can find a new point on our line P0123. With this point we can split out bezier curve into two curves.

- Curve 1: P0 [cp: P01] -> P0123 [cp: P012]
- Curve 2: P0123 [cp: P123] -> P3 [P23]

Now we can repeat the process to find the mid point of these curves etc.. Using this technique we can find the bezier curve to any level of accuracy. To get a greater accuracy we just perform more bisections and generate more points.



Subscribe to RSS - bensmiley's blog