Here is my completed map project for Virtual Entertainment:

The goal of the project was a large print with enough detail that it was still interesting when enlarged:

The project started out with attempts to import map files into Processing.

After overcoming this hurdle. I created a class for vehicles that “drive” around on the lines that represent roads on an image.  As the vehicles analyze pixels rather than vectors, this actually meant that I could import any black and white image to use for a map, and that the use of .e00 files or shapefiles was not essential (although I still ended up using them).

However, these vehicles followed the lines of the map so precisely, that the resulting image was simple the original map, but in different colors.  In order to introduce more variation into the equation, I used these “Follow Points” as the targets for a seeking behavior, and then built a tolerance into the seeking behavior.  What this means is, the new vehicles would try to point to the Follow Points, but would stop changing the direction if they within a certain range.  This produced a great variety in lines, simulating a street where vehicles could drive on any section of the road they wanted, as long as they stayed inside the boundaries of the road.

I decided the top-down view wasn’t as interesting as I wanted, so I moved the entire thing over to 3D. Before, I had simply had every vehicle leave a trail behind it by never clearing the background.  However, I wanted to be able to rotate the image in 3D.  To do this, I gave every vehicle a history array that would keep track of where it had been, so its entire light trail could be drawn every frame.  The p3D renderer in Processing really started to struggle here, so I switched over to OpenGl, which is much faster.  Rendering the map in 3D did add some visual interest:

At this point, I also set the project up so that it would render the entire project out as a PDF, meaning I could print it at any size.  This looked fine when it wasn’t enlarged, but when I rasterized the images at a large size in Photoshop, the strokes began to appear dotted, as apparently Processing doesn’t export strokes as being continously connected, but instead saves them as separate line segments:

pdf = (PGraphicsPDF) beginRaw(PDF, “pdf_lines.pdf”);
pdf.strokeWeight(0.25);

To remedy this, I set the pdf’s stroke settings in a way that Photoshop and Illustrator would render more nicely.  The following settings:

pdf = (PGraphicsPDF) beginRaw(PDF, “pdf_lines.pdf”);
pdf.strokeJoin(MITER);
pdf.strokeCap(SQUARE);
pdf.strokeWeight(0.25);

resulted in an image that was far less objectionable, unless zoomed to a very high level.

To make the map even more interesting, I created a simulation that tracked how often vehicles drove by a certain area, thereby creating a map of traffic density, which I figured I could roughly correlate to population in my simulation.

My eventual goal was to use this as a way of generating “cities” to match the “roads” that the simulation already created.  I successfully created a series of dots representing cities, with the size varying by population:

However, this, when combined with the lines from the roads, became too visually busy.  In fact, I liked the initial rendering with the vertical lines better.  This led me to try a number of other visualizations,a ll of which ended up looking terrible.  I eventually decided that I liked the original version with just the traffic lines better.  Thomas Edison said he didn’t fail a thousand times while trying to make a lightbulb, he instead discovered a thousand ways not to make a lightbulb.  Similarly, I could say I wasted a lot of time trying things out and then coming back to the original, but I’ll try to look at it as discovering ways not to make a map look good. Actually, even when I put it that way it still sounds like kind of a waste of time.

So I decided to refine the original lines image to make it more intersting.  To add a bit more variety, I rendered out several versions with different line settings.  I then combined several of these so that there would be lots of “detail” lines forming the major shapes of the roads, but several thicker and brighter “accent” lines would add visual interest.  I ended up sing three different line weights, which gave a visual variety that I was happy with.

I even ended up using the population circles as a way of varying the lighting on the thinnest lines, creating an interesting effect, and ensuring that my work on the population simulation did not completely go to waste.

A few more post production tweaks, and I ended up with my final print.

I’ll post the source code once I clean it up a bit.  I realize I’ve said that in several posts and have yet to post any of my code, but I also realize that probably nobody cares.

Here’s a brief tutorial on how to create a 3D visualization of a map in Processing, using .e00 files as source data. The end result will look something like this:

The height of the various bars is based on the number of urban areas in that region. Data is from the US Census Bureau.

There are 3 main things you have to do to create this:

1. import and parse the map data
2. draw and simplify the data
3. create and export 3D objects based on the data

1. Importing the Map Data

.eoo files are raw text files, storing groups of longitude and latitude coordinates:

Text files can be read into processing through the loadStrings function:

String lines[] = loadStrings("ua99_d00.e00");

This creates an array of every line in the file. Since .eoo put each coordinate on its own line, this is perfect for what we want. Now we can split up each line into the numbers it contains:

for (int i=0; i < lines.length; i++) {  //go through each line
	String[] c = split(trim(lines[i]), ' ');
	ArrayList vals = new ArrayList();
	for (int j = 0; j < c.length; j++) {
		float val = parseFloat(c[j]);
		if (!Float.isNaN(val)) vals.add(val);
	}
	//now we have an ArrayList of all the numbers for this line
}

We can see that lines with coordinates have 2 numbers separated by spaces, and other lines, between groups, have either less or more numbers. The coordinates are all we care about, so we’ll pretty much be ignoring all other numbers. To make things easier. I created a Point class, which holds a coordinate, and a PointGroup class, which will hold a group of Points:

class Point {
	float x,y;
	Point(float aX, float aY) {
		x = aX;
		y = aY;
	}
}

class PointGroup {
	ArrayList pointsList;
	PointGroup() {
		pointsList = new ArrayList();
	}
	void addPoint(Point aPoint) {
		pointsList.add(aPoint);
	}
}

Now, we can add to our number parsing code, so if a line is a coordinate, it adds it to the current PointGroup, and if it’s not, it ends that PointGroup and creates a new one.

ArrayList groups = new ArrayList();</strong>
for (int i=0; i < lines.length; i++) {  //go through each line
	String[] c = split(trim(lines[i]), ' ');
	ArrayList vals = new ArrayList();
	for (int j = 0; j < c.length; j++) {
		float val = parseFloat(c[j]);
		if (!Float.isNaN(val)) vals.add(val);
	}
	//now we have an ArrayList of all the numbers for this line
	if (vals.size() != 2) {
		groups.add(new PointGroup());
	} else {
		PointGroup currentGroup = (PointGroup) groups.get(groups.size() - 1);
	}
}

Now, we have all of the map data parsed into memory, and all we have to do to draw it out is connect the dots within each PointGroup.

2. Draw and Simplify the Data

Now, we can draw all of the data onto a BufferImage in memory. If all you want to do is draw the map to the screen, there is no need for a buffer image. However, since we want this file to eventually display in 3D, we can’t draw our points to the screen, so we create an Image in memory:

PGraphics buf = createGraphics(800, 600);

This essentially creates another canvas for you to apply any drawing commands to – a little bit like a MovieClip in Flash that isn’t being displayed on the stage. Now, we can cycle through our PointGroups, and connect all of the points inside. In this case, we are using beginShape(POINTS) because we don’t want the lines to connect, but normally you would just use beginShape(). I hardcoded in the boundary values I was using for the map of the US, but it would be fairly easy to make the map automatically scale to fit the image.

minX = -140;
maxX = -60;
minY = 5;
maxY = 60;
buf.smooth();
buf.stroke(255);
buf.noFill();
for (int i = 0; i < groups.size(); i++) {
	PointGroup aGroup = (PointGroup) groups.get(i);
	ArrayList points = aGroup.pointsList;
	beginShape(POINTS);
	for (int j = 0; j < points.size(); j++) {
		Point aPoint = (Point) points.get(j);
		vertex(aPoint.x, aPoint.y);
	}
	endShape();
}

Now, we can create a function that will basically pixelate this image, so that the color value of each square reflect the average color of that area (which in this case, reflects the number of populated places). We could create a new buffer image if we wanted to display the image, but since don’t we can just create a new pixel array:

int squareWidth = 5;
int squareHeight = 5;
int[] averages = new int[buf.width * buf.height];
for(int gx = 0; gx < buf.width / squareWidth; gx++) {
	for (int gy = 0; gy < buf.height / squareHeight; gy++) {
		int count = 0;
		for (int x = 1; x < squareWidth ; x++) {
			for (int y = 1; y < squareHeight; y++) {
				if (pixels[ x + gx * squareWidth + (y + gy*squareHeight) * buf.width] != color(0,0,0)) count++;
			}
		}
		// Saves as a color value, although we don't ever display it
		val = round(255 * count / (squareWidth * squareHeight));
		averages[gx + ceil(buf.width / squareWidth)] = val;
	}
}

3. Create and Export 3D Objects

Next, we can create the 3D objects that we want and export a DXF file. We will use the Cube class from the Brick Tower example by Ira Greenberg that ships with processing. The code simple goes through the array we just set up, and creates a stack of cubes of corresponding height:

for (int x = 1; x < buf.width / squareWidth; x++) {
	for (int y = 1; y < buf.height / squareHeight; y++) {
		int num = img.pixels[x +  (y * squareHeight + 1) * round(buf.width / squareWidth)];
		if (num > 0) {
			for (int i = 0; i < num; i++) {
				Cube brick = new Cube(squareWidth-1, squareHeight-1, 2);
				pushMatrix();
				translate(x * squareWidth, y * squareHeight, i * 3);
				brick.create();
				popMatrix();
			}
		}
	}
}

Then, we can output it as a DXF:

beginRaw(DXF, "output.dxf");
//code from above
endRaw();

Run this code, then open up Cinema 4D, and choose File > Merge, then select the output.dxf file that Processing created. Select the object and move it to 0,0,0 since it comes into Cinema somewhere way off the screen. You’ll notice the polygon faces look really weird – that’s because some of the polygon normals are reversed.

Throw everything inside a Connect Object to fix this(It may take a little while to process), then convert the connect object back to a polygon. Remove the phong tag, so that Cinema doesn’t try to smooth the edges of the cubes. That’s it! Throw some materials on it and render it out.

I’ll post up the source code once I get a chance to clean it up a bit.
Temporary Files

I created a processing application that parses and displays the longitude and latitude coordinates from e00 files and ascii GIS files, and then maps them to the the display space.  The resulting maps are a bit distorted, but recognizable enough to work with.

Also, I am looking into a few programs that will convert .shp files into e00 files, which would give me a lot more localized data that I could feed into the program – every street in a particular city, for example.

For my second Processing project in Virtual Entertainment, I am going to try to create a simulation which abstractly resembles a long-exposure aerial photograph of a city at night.

A road map will be generated or imported, and various vehicles, controlled by simple AI, will drive around, leaving light trails behind them.  These light trails will trace the shape of the roads, and, based on the popularity of the roads, lights will appear along the side, causing some areas to look more “built up”, while other areas remain dark.

My first step in researching this was to figure out how I would create the road map in processing.  Ideally, I want to use actual road maps of actual cities, rather than having processing generate a fake road map, so I looked into a few options for importing real street data.  The first option was to use the TIGER/Line files provided by the US Census Bureau.  I did a lot of research, but couldn’t find a good way of importing these .shp files directly into processing (although it must be possible, since Ben Fry did it).  The files are in binary format, so opening them in a text editor doesn’t reveal anything useful.  They appear to be mainly for use with GIS software.

After poking around a bit more, I found that the National Atlas offers much of their data in a RAW format for download as .e00 files.  I found even less documentation on how to deal with these files, but at least when I opened them in a text editor, the data was decipherable, and it is easy to pick out coordinate pairs.

I’m not sure exactly what to do with these points yet, or how to place each grouping in relation to each other, but at least this is something I can work with.  Plotting one set of points by hand resulted in a squiggly diagonal line – not very exciting, but presumably it will look more interesting when done on a larger scale.  Unfortunately, the file seems to randomly switch formats several times throughout the file, presumably storing different kinds of data.  This means I would have to decipher 3 or 4 different formats of RAW data, and figure out which was the actual road data I’m looking for.

I also found that the US Census bureau offers some data, such as zip code boundaries, in ASCII format, which is also readable in a text editor, and looks fairly similar to the e00 files:

This isn’t street data, but at least the format is consistent the whole way through, and it appears to be slightly easier to decipher.  This could lead to an interesting “USA as a city” project, where I create the simulation using zip code boundaries as streets in a city.  I may also look into the possibility of using other kinds of data, like airline flight paths, to generate my fake road map.  This could turn out to be interesting, and I would much prefer to make this project a visualization of some sort of meaningful data than to have processing generate fake random street layouts, which is currently my last resort.

The next step will be to import the e00 and ascii files into Processing, and have the program plot all the points onto an image, so I can see what I’m working with.

My group in New Media Team Project has created a blog to keep track of our progress.  Our project is to create an iPhone application that allows people to create drawings in 3D by waving the iPhone around in space.

Final Image

My first project for Virtual Entertainment uses Processing to visualize a mathematical system called a strange attractor.  Strange attractors create a cloud of points based on a quadratic map, a series of letters which the program translates into numbers which control its movement.  For example, the final image used the pattern “AMTMNQQXUYGA”.

While there are a huge number of combinations that can be created, most of them seem to move  fairly continuously in one direction, and don’t create an interesting pattern.  A computer program can figure out which patterns work and which don’t, but luckily my atrophied math skills got a break from having to figure this one out, as other people have already done it.

The basic attractor renders out a cloud of points:

Cloud of Points

I then explored several options for connecting the points, initially with straight lines:

Lines

Then with Curves:

Curves

At the same time, I modified the file to output a PDF, so that everything was in vector and can be scaled to any size.

Finally, I connected all of the points with a continuous spline.  Unfortunately, this narrowed my options for which attractors I could use, as it caused several patterns to look pretty lame:

Boring

Finally, I added color, by having processing grab colors out of an image file, resulting in the final image at the beginning of the post.  I also ended up with a few other images that I thought were cool.

I also explored some other options for this project, which didn’t turn out badly, but still need some more work.

[Download Processing Code]

References

• Strange Attractors at Wolfram Research Mathworld
• Julien C. Sprott on Strange Attractors

Here are some other cool images that came out of my Virtual Entertainment processing project:

I was exploring several options for my first Processing Project in Virtual Entertainment.  This was the other path I was exploring:

I’ll probably add some more info on how I did this soon.

In Virtual Entertainment this quarter, we are learning how to use the Processing programing language,which is designed specifically for creating interesting visuals ad generative art with code. Here are some of my very basic initial experiments: