HTML Canvas

Interesting development. The guys behind FreeCiv have developed a client using the HTML 5 canvas.

I remember looking at a proposal for canvas a while back. It was sorely lacking from the Tcl canvas (if only Tcl had made it as a web standard!).

But, if you can implement FreeCiv in it, maybe it has gotten better (or maybe FreeCiv doesn't use circles...).

Probably worth looking into. A NewStars web client would be nice.

NedOS on CD

Booting from a floppy is a good start. But there's just not much you can do with 1.44MB.

I looked into bootable CDs, and they are really nice!

• You get >600 MB of storage for your OS and app code + read-only data.
• Writable CDs are cheap and portable, potentially allowing you to run on any machine
• You don't risk messing up your hard drive (particularly the boot sector)

Best of all, the boot CD spec (El Torito) allows you to load multiple sectors immediately. That means no messing with BIOS read disk routines in the boot loader.

Of course, switching my boot floppy code to CD ran into multiple problems (all stupid ones).

• The boot code can load into any address. 0 will get you the default (7c00), but just to make sure I could write whatever I want when I need to, I set it to 7c00.
• Of course, it is a segment address, so I got 7c000, which is near 512K, not 32K, so I was executing uninitialized RAM.
• Then, when I fixed the segment, I forgot that my makecd program only writes those sectors for new disks, so I still had the old values.
• The boot floppy code is org 7c3e (to skip the Bios Param Block), but the CD code needs org 7c00.

It took longer than it should of, but I can now boot from CD!

New Stars Status

Hopefully things will pick up...

Here we have our first turn gen, and our first bug!
Our move code has place the destroyer and the probe in the same location, while Stars has them one light year apart. Bizarro. Must be a rounding problem...

I also had to fix the recent file LRU yet again. Hopefully it is correct now.

New Stars Status

Not exactly a blitz, but moving along...

Here is player 2. Note, I have fixed the title bar. I did the Stars turn by hand, but the New Stars turn was generated from Tcl!

$ cat orders_P2_2400.tcl
set owd [pwd]
cd ../../../gui/tcl
load ./nstclgui.so
source nsCli.tcl

set nsGui::playerNum [newStars $::ns_open $owd/tiny_sparse_P2_Y2400.xml]
buildMyPlanetMap
set hwPID $nsGui::myPlanetMap(0)

set fid [findFleetByName "Armed Probe #2443026239740"]
sendFleetToPlanet "Armed Probe #2443026239740" [findPlanetByName Pervo]
sendFleetToPlanet "Armed Probe #2443026239740"
sendFleetToPlanet "Armed Probe #2443026239740"

# get a useful scout
newStars $::ns_design $::ns_loadPre scout "Smaugarian Peeping Tom"
newStars $::ns_design $::ns_save

# build it
newStars $::ns_planet $hwPID $::ns_addAtIndex 0 0 1 $::designMap(3)

# set orders for max growth
for {set i 0} {$i < 100} {incr i} {
newStars $::ns_planet $hwPID $::ns_addAtIndex 1 1 3
newStars $::ns_planet $hwPID $::ns_addAtIndex 2 1 4
}

newStars $::ns_save $owd/tiny_sparse_P2_Y2400.xml

Pasting Tcl code is so much easier than C++, a lot fewer < and >!

I had to override the logic for "sendFleetToPlanet" auto-pick. It picks the closest planet, but I wanted to send the Armed Probe on a one-way trip to cover a specific set of planets. The Smaug will cover closer planets.

I had to adjust the speed of the probe in Stars, but the New Stars auto-pick got the speed I wanted, very nice!

New Stars does not show what will finish, that is a very hard problem. I will have to address it at some point, though...

New Stars Status

Let me start by saying the gcc maintainers are jerks.

I have heard this accusation often on comp.arch, but I was skeptical. Now I know it's true.

If you doubt, read this.

That was back in October, there is still no solution for Cygwin (as far as I can tell). My latest updates pulled down the latest versions, and now I can no longer build NewStars.

I had to back rev to gcc-3 (as mentioned in the post).

Which brings us:


Ahh yeah! Stars versus New Stars, head to head!

Turn 1, player 1 of the test game.

Obvious differences:

1. Stars and New Stars have the origin of their graphics inverted. This creates a "mirror" effect when looking at the universe (not going to fix)
2. Stars puts some useful info in the title bar (will fix ASAP)
3. I am missing the planetary mineral concentrations (will fix, eventually)

More soon!

P.S. Interesting factoids

1. Stars adds 1000 to x and y
2. I used Tcl to verify the structure of the master xml

package require tdom
set doc [dom parse -channel [open tiny_sparse.xml]]
set root [$doc documentElement root]

set univP [$root selectNodes //UNIVERSE_PLANET_LIST/PLANET]
set playP [$root selectNodes //PLANET_LIST/PLANET]
puts "#\tX\tY\tName"
foreach p $univP {
set name [$p selectNodes string(NAME/text())]
set id [$p selectNodes string(OBJECTID/text())]
set x [$p selectNodes string(X_COORD/text())]
set y [$p selectNodes string(Y_COORD/text())]

incr x 1000
incr y 1000
puts "$id\t$x\t$y\t$name"
}

That creates the "Universe Definition File" (Report->Dump To Text File->Universe Definition). I have a longer version which checks the integrity of the player maps against the global map.

We're On Pluto

I have been catching up on the newsgroup rec.arts.sf.science. There was an interesting interchange on Project Pluto.

Now, when some people think of Pluto, they think of the ninth planet (yes! a planet! My Very Excellent Mother Just Served Us Nine Pizzas). More, probably think of Mickey Mouse's dog.

Of course, Pluto is the Greek god of Hades/Death/the Underworld. That is probably the inspiration for this thing:

• 75 foot long; 60,000 pound
• nuclear ramjet, cruise missle bus
• 26 nuclear warheads

The second point needs some explaining.

• ramjet - flies by compressing incoming air, this requires a fair amount of air to work at all - end result - it cannot fly slower than the speed of sound (it would be launched using solid rocket boosters, and could never land)
• nuclear - the incoming air is heated by being directly exposed to an unshielded, operating nuclear plant. This means the exhaust is highly radioactive.
• cruise - designed to fly close to the ground (about 1000 feet)
• missle bus - a missle that carries missles (the 26 warheads)

This thing (designed circa 1960!) would follow terrain using pre-constructed radar maps, shattering glass and damaging walls and roofs as it flew to its target - all the time irradiating everything it flew over. It would then pop out a nuke, which would detonate seconds later (Pluto flies just under one mile per second, so it is already miles away). Once it was out of nukes and losing power, it could be crashed into a secondary target (60,000 pounds at Mach 4, plus highly radioactive).

Magnum 360 said: "Thank you very much thats exactly what I was looking for. Wow thats amazing it had the possiablity of staying up in the air for months, to bad the project was stopped."

To which, Mike Ash says: "Too bad the unshielded and unfiltered airborne nuclear reactor with no ability to land after launch was stopped??!!"

This is pretty incredible work (terrain following using vacuum tubes?). Also, the sheer gall of the thing (spewing out radioactive exhaust). Of course, this was the same age which planned to blast a mountain into space using nuclear bombs!

You have to admire their sense of scale and boldness. Today, we are so safety conscious - we'd never do anything approaching this scale. It seems like there should be some middle ground...

New Stars Status

Well, looky here!



Here is the "Merge Fleets" dialog! (I know the spacing is off) It works! You can merge ships between fleets. I still don't have "Split All" and "Merge All", but they are basically macros that should be easy. All the core functionality is done.

I also fixed the "Goto" button in the "Other Fleets" area. It was really bugging me...

We are now ready to play! (except for fleet to fleet cargo transfer, but who ever does that :)

Git 'er Done

I've finished looking over the "Why Git is Better Than X", and the propaganda pages from Bazaar and Mercurial (propaganda is good, you need to say why you think your system is better).

I think I had previously decided on Bazaar. I was really worried about Git on Windows, and Git's main features "Cheap Local Branches" and "The Staging Area" didn't impress me at the time. Bazaar has some really nice support for implementing different sharing schemes on top of the distributed model.

But now, I think Git is the way to go.

In my day-to-day work, I have at least three "local branches" in svn. They are:

1. the working trunk - attached to the trunk, with all the changes I am thinking about making
2. a clean trunk - this is my "Staging Area" where I copy changes in before commit
3. an old copy of the trunk - for comparing really old revs (because history is not available offline in svn)
4. a personal branch - for long term projects to make major changes
5. a release branch - for when we are doing a public release

Each local repo can take up to 2 GB (if I need to build the executable). Wasting disk space isn't the biggest worry. It takes a long time to set up these repos, and a lot of work to keep them all in sync and up to date.

I really overlooked the Staging Area.

My concern was that I wouldn't be able to use it, because I need to test any changes before pushing anyway. Of course, I realized that I just need to push to my clean area, test there, then push to the "main" repo (as much as any repo is main in a distributed SCM).

Stuff I've read lately

"A is for Alibi" (Sue Grafton)(audio) - This is the first in the "Alphabet Murder" series. It's interesting to see how Grafton has developed as a writer.

And it is good they replaced this narrator.

She is terrible.

She has two tones - monotone and slightly deeper monotone. For an audio book, you need at least two distinct tones, so you can figure out when the speaker changes in a dialogue (basically, to denote the paragraph breaks). Distinct voices for each character and one for actions is a plus, but not absolutely necessary.

A lot of the time, I couldn't figure out who was talking - even if I was paying attention. And if distracted (say, by a crazy MD driver), it was easy to get lost. Sometimes it was hard to tell if someone was speaking or if something was happening.

That said, the story was ok. Grafton drifted some into "romance novel" territory, which she is obviously terrible at. Good to see she has dropped it in the later novels.

New Stars Status

So close now!

It's been a while since we had a Windows screen shot. See, it still looks the same! :)

I had to scp over the turn file from Linux for debugging, because I don't have a split case on Windows, and debugging through VNC can be a pain...

This looks like the normal split case, but look carefully...

The original fleet id is 234806..., now the fleet id is different. I did a split, where all the ships move into the new fleet (and the old fleet gets deleted). That was to test the fleet deletion code, which is needed for merge to be useful.

Now I can do merging.

Give in to the Darcs Side

Continuing from my last post, I did want to talk about SCM I will not be considering:

Darcs

• Written in Haskell, considered the poster-program for Haskell. Haskell is a functional language, similar to ML. Given the horror of ML, and the behavior of Darcs - I am not motivated.
  
• Operates on a "theory of patches". Finding the right set of patches, and applying them in the right order. I don't believe it is possible for a machine (currently) to handle this level of complexity. Heck, I don't know that I can handle the complexity!
• Ate our repo, twice. Losing the repo is an unforgivable sin for SCM.
• Was really slow. They are supposed to have fixed the number of "exponential merges" (nice). But it was slow all the time. The exponential merges were just insult to injury.
  

BitKeeper

• Commerical software. No way I am going to pay for software for my personal projects.
• Also ate the repo. Maybe twice.

SVN

• No distributed model
• Logs and history are not available offline
• While creating branches is now easier than CVS, merging them back is still just as hard (this is a hard problem, so I don't blame them)

Source Control to Major Tom

To a coder, nothing is more precious than his code. Source control management (SCM) is all about keeping your code safe. Not against disk failures, so much as against bad edits ("what was I thinking!" !). It's also a good place to stash some forms of comments (not what the code is doing, but why it was written). Mixing this into the code can be distracting. SCM also provides a nice log of when features were added ("it's been two years!?").

I had my first encounter with source control as an intern at Intel. Our files were under CVS (concurrent versioning system), with some binary files directly managed with RCS (CVS started as a wrapper around RCS - all those funny ,v files).

CVS is ancient (1990), and works really well. You have to be careful with merging conflicts, tagging takes forever, and branching is so confusing no one ever used it. You have to stretch the meaning of "really well"...

Eventually (about ten years ago), some people decided to improve on CVS. That project is called Subversion (svn). I didn't use Subversion until about two years ago. SVN is a big improvement over CVS. The main thing - being able to do a diff against a network repo while not connected to the network. Also, SVN is much better with merging. Tags and branches use the same mechanism, which is cheap and easy. Very nice.

I used CVS professionally until about 2003. That was when I was exposed to the Great SCM Wars - between minions of Darcs, and keepers of BitKeeper. I will have to rant about them sometime. Suffice to say, that no man has ever fought with them and lived! The bones of many projects lie strewn about their lairs. So if you do doubt you courage, come no further. For death awaits you all; with nasty, big, pointy teeth.

Ok, so I still use CVS for my personal projects. I would like to switch over to something new, probably "distributed". There are three alternatives:

1. Git - created by none other than Linus of Linux fame. He too was a victim of BitKeeper, so he is familiar with pain. He is a really smart guy who churns out quality software at a prodigious rate. He also tends to be a little quirky, and he is (ultimately) writing software for himself, so there is no guarantee I will get the same returns...
2. Mercurial (hg) - apparently started about the same time as Git (2005), for the same reason (holding the Linux kernel tree)
3. Bazaar - the newbie in this line up (2007, and new is bad, like in compilers - new means buggy and untested). But, I am willing to give it a shot.

I had previously compared these three and decided on a winner. Sadly, I can no longer remember which one I picked. And things have changed some since then. At the time, Git was basically Linux only (there was a Windows hack by some guy who was trying madly to keep up). Now, I can get Git as a Cygwin package.

I will have to go through again and compare. I don't think SVN is in the running, as it does not have a distributed model. I like the distributed model, because it allows me to push code from one machine to another without having to worry about which one is the "master".

The Bootstrap Problem

When you turn your computer on, it just boots. You take it for granted. It says "Windows loading" or "Grub", or "L..I..L..O".

But a lot of work goes into getting your computer to boot.

The main problem is the processor just runs programs. And it needs a program to tell it how to load programs. But you need to load a program, to run the program...

The solution is a program which lives at a particular address in permanent memory (ROM). This program is called the BIOS, and it has a really remarkable history going back to the original IBM PC.

The BIOS finds a boot device (usually floppy, harddrive, or CD-ROM, but now usually includes USB). It then loads the "boot sector". This is a 512 byte block of memory containing code which is placed at address 0x7c00. It then jumps to 0x7c00, and you're off!

In my NedOS musings, I considered using Grub as a boot loader. It handles a lot of stuff, and allows you to build your kernel like a regular program. In the end, I rejected it, because:

1. I wanted to learn how to do a bootloader
2. I didn't want to be bound to ext2fs and Elf (or worse, FAT32 and PE)
3. You really need to build a cross compiler to use it
   
4. Grub seems bound to 32bit kernels
5. Grub is 3 stage, I wanted to see if I can get into 64bit mode in 1 hop

The fifth point is the most interesting. Your first stage is the bootsector (512 bytes, really 448 on a boot floppy). That's all you've got to get going, it is really cramped! But loading a second stage requires disk accesses and some sort of file system. I don't want to think about file systems right now, and I don't want to pick an existing one and get stuck with those assumptions.

After most of the day hacking, I have a minimal 64bit bootloader, in 170 bytes!

Now I need to figure out what BIOS info I want to extract before kicking into protected mode, and how I want to find the kernel.

Ugly/cool tip:
You cannot get to 64 bit mode in one step. You have to go through protected mode (because the bit for long mode is in a different register than the register used to enter protected mode - jerks). Further complicating things, the code descriptor for 32bit code is incompatible with the descriptor for 64bit code (nice work guys).

So what I do:

• set the descriptor for 16bit protected mode code (yes, the original 286 style)
• kick into protected mode
• set for long mode (I'm now in a really tenuous position, but interrupts are off)
• flip the descriptor bit for 64bit code
  

I could switch to 64 bit mode now, but I turn on paging first, just for good measure. Then with a far jump, I'm into 64 bit/paged mode!

Debacular

When something turns out really bad, you say it was a debacle. If you're in the middle of something that you know is going to end badly, you say the whole process is debacular.

So, a post on comp.arch led to a Google search, which led to a website called OsDev. Now, I have a directory in my projects area called NedOs, which I have been trying to figure out what to do with. There's really not much stuff in there, but it seems a shame to delete it. So, why, not, you know, write an OS? It's not like I'm hugely busy in every dimension, right? (mmm, debacular already)

I broke out my Intel software developer manuals, but they are all so old, they don't have 64 bit stuff. I downloaded the new one, and was greeted by this:

Now, that is debacular!

For anyone who is thinking of writing their own OS, this is the flow chart for enabling "Long Mode" (that is - X86-64, AMD64, EMT64T, or as Intel has now started calling it IA32e mode!) (technically it handles all the modes)

After staring at it for a while (I just want 64 bit mode with paging, is that so wrong!), I realized this could be three sentences:

1. Enable PAE
2. Enable LME
3. Enable PG

You could have an optional fourth sentence - "The first two steps can be done in any order."

So, this picture is worth, like, -994 words.