Platform Masters - Will you be the world's next platform master?
Date: June 18, 2014
Screenshot #541: With the trees sorted out, I wanted to know and make sure they were good. So, I wanted to test some worlds that I knew needed trees, were easily visible, and didn't have many. Ronnis is a great example of that. In the grassy areas lining the Keveran River, trees are present. In this case, there are 76 total unique trees, 55 on the far side of the river. This view shows the trees quite well. They definitely make Ronnis look much better. As of this time, Ronnis is still not fully finished yet - I still have those 3 objects in the foreground to do.
Date: June 22, 2014
Screenshot #542: Jeremy's House is now forested... sort of. There are plenty of trees scattered around throughout the neighborhood, but they have a very low density compared to what's otherwise available. There are 3 key types of trees used in Jeremy's House. The coloring for all the trees are set in POV-Ray. The variations in the color come only from the lighting which is generally sufficient enough.
The most common type of tree, taking up a bit more than half of the unique trees in this area, is one I don't know what the species is. The reason for it being unknown is mainly in that I customized it on my own through fiddling around and experimenting in Arbaro, trying to create something that had dense foliage. These are best seen in the screenshot above, for Ronnis.
The next most common type of tree, using a bit less than 1/3 of the unique trees, is the sassafras, a type of tree that is completely unfamiliar to me but liked the looks of. From the default settings, I only raised the quality since the default quality settings are quite low. The sassafras can be seen behind the closest house very slightly to the left of Jeremy Kaplan himself in the center of the scene.
The least common, still using about 1/6 of all the unique trees, is the weeping willow. This is one of my favorite types of trees. From the default settings, I raised the quality as usual but there were surprisingly few leaves for what I saw in a Google Images search (so I can get the colors) so I had to add a lot more leaves. The default leaf quality was also rather low so I raised that as well. Despite POV-Ray parsing 93 million tokens per tree (whatever that means), the weeping willows take about 80 seconds each to generate, and that includes all manual input that I don't know how to automate. This is because of the huge amount of detail they have. This screenshot really shows the awesome detail with the weeping willow, found in the far rightmost area of the scene.
You're probably thinking that the use of extremely high quality would have a severe impact on the frame rate. By prerendering the trees, which can sometimes take over a minute for a single tree, I'm taking out the complexity. Instead of having to draw several million leaves 60 times a second, a simple bitmap image is all that has to be drawn. Thus, the only impact the trees have on the frame rate is that it requires a draw execution, a few pixels need to be drawn with alphablending, and a bit of extra VRAM. Whether I use a very low quality tree that's 320x384 or a very high quality tree that's 320x384, it has no effect on the frame rate. So, it makes sense to push out as much quality as possible, maximizing the detail. This is how the weeping willow in the foreground, with all that detail it has, still allows for a 500+ fps frame rate. Compared to earlier screenshots of Jeremy's House from before the trees were added, it's unchanged.
Date: June 26, 2014
Screenshot #543: Ground decals are just that - decals. They are very rich in detail and almost anything is possible outside being able to vary on the Y axis (the height). Unlike Ronnis where the city is in the foreground area, ground decals often have cities in them. The process for adding these involves using the premade decals that Ronnis has for the far area. I scale it then tile it as needed. From that point, the creativity comes in. Here, I trace the boundary of the city. There are 2 ways of doing this. In this case, this is my chunking method where I take out entire city blocks. In the other case, I just draw a single 1-pixel-wide line all around the area. In both cases, I simply trace around the outer edge of the area where I want the city to be. At first, I disregard the minor roads and focus on where the freeways and county/state/providence roads are and work from these. It's only once I've laid out the cities that the fine tuning of the roads are done. I make the city texture transparent so I can see where the coast is so that buildings don't end up underwater or out in the middle of the ocean.
In this case, I'm outlining Yurisin, a city of around 130,000 residents. On the far right side of the map, there's also Yurisin Beach, a huge beach that has a 350-foot span of sand for the beach itself. You'll be able to see both very well from the tutorial level with the spring sampling area.
Date: June 28, 2014
Screenshot #544: This puzzle has bugged me for several hours on this day. I can get Arbaro, the tree generation program, to generate a tree through the command prompt (cmd.exe) but I can't get it to work in my own program despite having exactly the same command string. What's more is that, out of 4 other programs I've tried, 4 of them could be executed through my own program: cmd, dir, my hex editor (XVI32), and Virtual Dub. They all executed without any trouble. Only Java refuses to execute for some unknown, unexplained reason. This has been an issue I spent 4 hours fighting and getting nowhere fast.
You can see the code I have behind those 2 windows. The window in the top right corner is command prompt, as hinted at by the title bar. The window just below it is my own automation program I use for pretty much everything. Behind all this is Microsoft Visual C++ 2008 Express. The code I'm using for this (and all the commented out sprintf cases for all the many trials I've done to figure out what's going on) is visible. Note how I'm using sprintf to stitch together the string to form the command. Note how I'm also using printf to display, visually, what that string's contents actually are. With this full command known, I put it in command prompt and it works just fine. Do it through my own program and it doesn't work at all. It's exactly the same command but I'm getting 2 totally different results. As of the time this was posted, I'm still drawing blanks as to what the actual problem is.
Date: July 1, 2014
Screenshot #545: It took several days of combat but I finally got the tree generation program working. This is what it looks like. Don't let the command prompt window fool you. This window is mainly only intended to let me know know what the status of the program is. Since I can literally type this description while this goes on, I'm making rapid progress in multiple areas. How did the problem get solved? Here's what's going on:
1. I set some parameters to let the program know what the first tree to do is and also how many to do. There's 2 other parameters as well - an image size and a target world. The target world determines the variety of trees available. The image size depends on the Z distance (the scaling). The very closest of trees use 4096 pixels as the image height which turns into about 3100 after autocropping). Farther out, the output image size gets much smaller. At 1 SU, the trees use 4096 pixels as the height. At 32 SU, far enough that the tallest of trees (the eastern cottonwood in my case) are only 24 pixels tall but I output the image as 128 (autocropping it makes it about 100). Making the output layer bigger allows for antialiasing effects and also some flexibility in the resulting tree size.
2. The program sets some initial parameters such as the image size as to be included in the seed, the seed itself, the species name, the stem and leaf colors (stem also includes the trunk), and whatever else is needed.
3. The program then generates the scene file reading from the source with placeholders I've added in that always start with the "@" symbol to let my program know where to replace something such as the seed or the species name. The scene file is then saved.
4. Execution of external programs follows. First, it's executing Arbaro to get the tree to generate based on the parameters set earlier. The program is set to having the "below normal" priority as I only intend on this being a task that runs in the background. While the tree is generated, my program waits. Not waiting for it to close results in POV-Ray referencing data that's either outdated or currently being modified. The INC files that Arbaro outputs can be very big, pushing past 300 MB for the weeping willow and spruce trees. So, to make the most of the disk space I have, I replace the INC file. The "CreateProcess" function got Java to actually work.
5. Lastly, POV-Ray is executed to render the generated tree. Once the tree has been generated, POV-Ray comes in to render it based on the settings determined earlier in the loop. Like with Arbaro, I have to have my program wait for POV-Ray to finish rendering before I go on to the next tree. While I can't figure out how to get the window to stop appearing, I at least got it so that it doesn't get in the way. This is done by simply minimizing the program as soon as the window comes up. To stop the focus loss, you have to set this in POV-Ray's options. To stop the output window from appearing, you just use the -D command line option. With this set up, I can type this long post without interruption.
6. Once the tree is rendered, steps 2 through 5 are repeated for however many trees there are needing to be rendered. Since I have this as "below normal" priority, other tasks I do in the foreground such as working on the ground decals or interacting on Facebook can be done without any significant impact on their performance (only hard disk activity will result in a reduction in performance).
Just think: from this long battle, I can now see green, earn green, and generate green all at the same time. See green is from seeing the amazing output this has, along with seeing real world grass and trees from doing my cleaning job. From doing my job stuff, I'm earning green. While I'm doing my job stuff (or other things like taking a shower, eating, shopping, etc.), I'm generating green. This will save several months of time off PM's development time.
Date: July 9, 2014
Screenshot #546: At first, the tree generation program was generating one tree every roughly 16 1/8 seconds. Upon noticing that the HDD activity light wasn't on much and that only 13% CPU usage was being reported (from 1 core), I thought I could make use of multithreading to drastically speed this up. Unlike with PM, my automation/batch processing program is immune to the lower system requirements that PM uses. This means I can use multithreading all I want, or features that are only first available in Windows 7. Upon further research, I've seen that the CreateThread function is not XP and later, it also works with Windows 98 so I also have multithreading in Platform Masters figured out.
Just for reference, with 2 threads in use, I found that I was needing about 9 seconds per tree on average. With 3 threads, that dropped to about 7 seconds per tree. With 4 threads, it dropped to 5 seconds per tree. Why not go more? At this point, I've run out of physical cores and the HDD is almost constantly going. Also, it's quite memory intensive so unless I install more RAM, I wouldn't have enough memory (and 12 GB is what I have). Note that java.exe is using over 1 GB (and it can push 1.7 GB with the weeping willows, 1.4 GB with the spruces). In addition, thanks to being unable to stop the POV-Ray window from appearing in the task bar, it's quite difficult to use the task bar as I often miss the window I want. To help combat the missing, I limit the closing of windows and use the X in the corner instead of right-clicking on the window in the task bar like I usually do. Why does Windows Task Manager have "5" showing for the total threads? That's because I've spawned 4 threads to bulk generate the trees. The other one is the original, main thread. It waits until the other 4 have finished generating the trees.
My approach to multithreading, in this case, is to have each thread process only specific tree ID numbers. Thread 0 gets tree numbers 1, 5, 9, 13, etc. Thread 1 gets tree numbers 2, 6, 10, 14, etc. Thread 2 gets tree numbers 3, 7, 11, 15, etc. and thread 3 gets tree numbers 4, 8, 12, 16, etc. This way, there is no chance of a conflict between threads. It's the easiest approach. If I were to use 5 threads instead of 4, then thread 0 would get trees 1, 6, 11, etc., thread 1 would get 2, 7, 12, etc., and so on.
With the tree generation program multithreaded, what used to be about 3000 trees in a day is now about 10,000 trees in a day. With yet another program to autocrop all the trees then another to generate a forest texture based on these trees for use in ground decals, it's looking like I'll need a grand total of anywhere from about 210,000 to 240,000 total trees.
Date: July 10, 2014
Screenshot #547: It used to be 144 SU wide and now it's only around 10 SU - the Keveran River in Ronnisa Plains. In this case, it's pretty much done, though I'm debating on whether or not to get rid of the road I mixed in (the shadows on the far right hint to it). This update significantly enhances the far area of Ronnisa Plains. Ronnisa Plains isn't done yet though - I still have a lot of trees left to be added. The odd colored areas along the river are picnic tables and tents.
Date: July 10, 2014
Screenshot #548: Due to an oversight where I had the trees a lot brighter than they should've been, the forest texture blended in too much with the grass. I did some simple adjusting to offset this just so I can get a preview. Other than correcting the forest texture (that's another 4 1/2 days or so), and debating on the positioning, these ground decals are done. That big city on the far right is Yurisin, home to 1/4 million residents. The ocean looks a bit better than it did before and the farm fields are no different from before other than having been rerandomized for their colors and cut-out areas where forests and ponds are. (Of course, with the trees being too bright, I also need to redo the roughly 320 trees I've already done but at least I could get some good optimizations going considering I know that plenty is possible, especially in Jeremy's House.)
Date: July 23, 2014
Screenshot #549: Keveran Forest, thanks to the mist, is shaping up to be a very tricky world to plan. It's, perhaps, the most draw intensive world of them all. It's also quite complex, having the most standard layers of all worlds. Several things need to be carefully considered. The biggest is memory as I'm pushing very near 96 MB. The second is the draw count which will probably push 2500 or more. The third is the total pixel draw count which could exceed 40 million pixels per frame. Basically, I really must focus strongly on optimization, squeezing all that I can out and there's a lot of optimizations I could make.
In this case, I needed to generate 25,151 trees to get just the layers from 32 to 63 3/4 SU. It would've been closer to about 40,000 had I not kept a close check on the memory usage and that's still only just the farthest layers. Because I could integrate the fog into the trees, I only have to do one draw, and it doesn't use any extra memory. Then of course, there's the hills. I could simply integrate the trees in with the hills which already have the fog integrated in with them. Again, no extra draws, pixels being drawn, or extra memory. I'll take up more disk space in the final product though, but that's only because it doesn't compress as good.
Date: July 25, 2014
Screenshot #550: Welcome to the organic industrial center of photosynthesis! Keveran Forest is finally forested for the first time ever, using realistic-looking trees. More work remains to be done, such as getting the Teigram River to be varied, but, so far, this is meeting my expectations. There are literally 60,000 trees visible here. You'd think that this would have a huge negative effect on the frame rate, right? Nope. Having over 500 fps is no joke! This is what prerendering is good for. The layer at 63 SU, for example, has about 440 trees visible. The layer at 18 SU has about 126 trees visible. Each layer is spaced only 1/4 SU apart making for a huge quantity of layers and an insane number of trees.