June 8th, 2008 Michael Clarke
At last it has happened! I have been putting off writing my memory allocator for MouthOS for some time. Up to now I’ve been able to survive without one by simply not having any ‘undefined’ data structures - everything in the kernel was a known, static size.
So after a interesting Solaris Advanced Crash Dump Analysis course last week (which has really put me back into my OS development mood) I decided it was time to make a move and I spent a large portion of Friday night working out how to allocate memory without overwriting the existing kernel, and at last by about 2am in the morning having messed around with the linker and some cleverly placed assembly labels I finally was able to kmem_alloc() some memory.
At this point I decided I was finished and went to bed. The next morning I awoke, pleased to have kmem_alloc() working, or so I thought. However, as the day progressed and I allocated more and more pages of memory I started to see some strange things happening. Why did that character on screen just go red? I never put a start on the screen…
After a fair amount of time debugging the issue I managed to isolate the problem. My array of free_pages_t structures were located, not at the end of the kernel stack as I thought (which is currently 8K), but in the last 1K of the stack. Ultimately the problem was a off by one (er, or 1024 depending on how you look at it) error in my programming. At last I was able to kmem_alloc() pretty much as much memory as I wanted - but how to free.
This proved to be very easy. I simply modified my page_map_t structure to contain a grouping id. As such, when we call kmem_free(void *) we do the following to first make sure that the address we got was on a page boundary.
while ( (addr % (PAGE_SIZE * 1024) != 0 )
addr--;
while ( page->addr != addr )
page = page->next;
Now we’re on a page boundary we can just loop through the pages whilst the alloc_id’s are equal and set the flags to PAGE_FREE…
int alloc_id = page->alloc_id;
while ( page->alloc_id == alloc_id ) {
page->alloc_id = 0;
page->flags &= PAGE_FREE;
page = page->next;
}
With that working I then spent a bit more time perfecting and tweaking and now MouthOS is pretty good at allocating and freeing memory - at least, I think it is anyway. In the end it turned out not to be so difficult - you just have to get your head around the whole idea of allocating memory, before you have an allocator. When I thought about it this ended up not to be such an issue after-all…. Here is how we can do it.
- You know the point where your kernel loads.
- You can work out where your kernel ends by putting a label in its own section, and then using the linker to put that section at the end of the kernel.
- You now know (from the placed label) where a safe place in the memory is to write without overwriting the existing kernel, so write all your required memory structures, of which you can calculate how many you need from the amount of memory in the system.
- Finally,go back to your memory structures that you’ve just written and mark all the kernel space as allocated, including the space the memory structures take - which you know as you know the size of a memory struct, you know where it starts, and you know how many you’ve got.
Easy 
You can find the code for the new MouthOS memory allocator at: http://www.michael-clarke-blog.com/trac/MouthOS - if you’re remotely interested :p
Until next time.
Posted in MouthOS, Operating Systems, Programming | No Comments »
May 16th, 2008 Michael Clarke
It’s taken me some time to gather the enthusiasm to do this - but I’ve finally moved my MouthOS source over to my web server and installed trac. So, you can view the MouthOS wiki and source at:
http://www.michael-clarke-blog.com/trac/MouthOS
The older versions are in /proto. Kernel version 0.0.1, 0.0.2 and 0.0.3 are broken with some pretty big bugs (for such small programs) such as triple faults and Kernel panics.
The most stable version at the moment is version 0.0.03 in /releases.
The stuff in /trunk is the current development with all the new HAL (Hardware Abstraction Layer) stuff. I’m still in the process of porting a lot of the proto-0.0.3 and releases-0.0.3 stuff to /trunk - so this is pretty useless at the moment 
But you’ll get the general idea as to where the project is heading from /trunk in terms of the HAL, Memory Management etc.
At some point I’m also going to add anonymous svn access so you’ll all be able to checkout the source code and have a play - but at least for the moment you can view the code, the wiki etc.
For the moment - enjoy
Posted in MouthOS, Operating Systems, Programming | No Comments »
April 5th, 2008 Michael Clarke
The other day I wrote a small tutorial on getting dynamic class loading working for Java. In that post I mentioned that it could also be done in C# - but that it was more difficult, and from my personal experience, there is less information available about it - so here is how to do it!
First, as with the Java, we need to create our API. However, unlike the Java there is an additional requirement - a ‘module manager’ class needs to be within the API. As such you will need to create these two files…
using System;
using System.Reflection;
namespace com.michaelclarkeblog.api {
public static class ModuleManager {
public static Module getInstance(String fileName) {
/* Load in the assembly. */
Assembly moduleAssembly = Assembly.LoadFile(fileName);
/* Get the types of classes that are in this assembly. */
Type[] types = moduleAssembly.GetTypes();
/* Loop through the types in the assembly until we find
* a class that implements a Module.
*/
foreach (Type type in types) {
if (type.GetInterface("Module") != null) {
/* Create a new instance of the 'Module'. */
return (Module)Activator.CreateInstance(type);
}
}
return null;
}
}
}
using System;
namespace com.michaelclarke.dynamic.api {
public interface Module {
String getText();
}
}
Once you’ve got the two files you can compile them into a library….
api $ mcs -t:library Module.cs ModuleManager.cs
api $ ls -a | grep .dll
Module.dll
api $
Now that we’ve made the API the next thing to write is the application that is going to use it for loading modules. This is a very simple application…
using System;
using com.michaelclarkeblog.dynamic.api;
namespace com.michaelclarkeblog.dynamic.application {
public class Application {
public static void Main(String[] args) {
if (args.Length > 0 ) {
Module module = ModuleManager.getInstance(args[0]);
Console.WriteLine(module.getText());
} else {
Console.WriteLine("Usage: mono Application module_name");
}
}
}
}
Now we can compile our ‘Application’ against the API…
application $ mcs -reference:../api/Module.dll Application.cs
application $ ls -la | grep exe
-rwxr-xr-x 1 michaelfclarke michaelfclarke 3072 Apr 5 19:32 Application.exe
application $
And now finally we can write a couple of modules!
using System;
using com.michaelclarkeblog.dynamic.api;
namespace com.michaelclarkeblog.dynamic.modules {
public class Mike : Module {
public String getText() {
return "Hello from Mike's Module!";
}
}
}
using System;
using com.michaelclarkeblog.dynamic.api;
namespace com.michaelclarkeblog.dynamic.modules {
public class Another : Module {
public String getText() {
return "Hello from Another Module!";
}
}
}
Compile the modules…
modules $ mcs -t:library -reference:../api/Module.dll Another.cs
modules $ mcs -t:library -reference:../api/Module.dll Mike.cs
modules $ ls -la | grep dll
-rwxr-xr-x 1 michaelfclarke michaelfclarke 3072 Apr 5 19:36 Another.dll
-rwxr-xr-x 1 michaelfclarke michaelfclarke 3072 Apr 5 19:36 Mike.dll
modules $
There is just one last thing we need to do before we can run our application. We need to copy the Module.dll API from the api directory into the application directory…
modules $ cd ../application
application $ cp ../api/Module.dll .
And now lets try running our dynamic application…
application $ mono Application.exe ../modules/Mike.dll
Hello from Mike's Module!
application $ mono Application.exe ../modules/Another.dll
Hello from Another Module!
application $ mono Application.exe
Usage: mono Application module_name
Posted in C#, Programming | No Comments »
April 3rd, 2008 Michael Clarke
It’s been quite some time since my last post. This is down to a number of things - the first is that I’ve been really busy and so haven’t had that much time to write a blog post. The second is, with being so busy with mundane things, I’ve not really had anything worth while blogging about.
However, today I was on the phone with a friend (Anton) and our conversation slowly progressed to Dynamic Class Loading, specifically in Java. This made me remember that when I first heard about this technology (about three or four years ago) there wasn’t that much information available on the internet. So, I decided that I’d write a little (or perhaps not so little) post on how to go about getting Dynamically Loadable Classes in Java - it really is pretty easy.
The first thing you need to define is an interface for your modules. Mine is just going to be something simple that had a getText method to get some random text…
package com.michaelclarkeblog.dynamic.api;
public interface Module {
public String getText();
}
You can now compile your API…
~/src $ javac com/michaelclarkeblog/dynamic/api/Module.java
~/src $
The next thing to do is to write your main application that is going to do the module loading. Mine is going to be a simple main method that accepts a java class name as a run-time argument.
package com.michaelclarkeblog.dynamic.application;
/* We must import the URLClassLoader and also our API. */
import java.net.URLClassLoader;
import com.michaelclarkeblog.dynamic.api.*;
public class Application {
public static void main(String[] args) {
Module module;
if ( args.length > 0 ) {
try {
/* Create a new instance of the module.
* Think of this as:
* Module module = new Module()
* Except we're doing it at runtime
*/
String path = "com.michaelclarkeblog.dynamic.modules." + args[0];
module = (Module)Class.forName(path).getConstructor().newInstance();
/* Run the modules 'getText()' method defined in the interface. */
System.out.println(module.getText());
} catch (Exception e) {
System.err.println(e);
}
} else {
System.out.println("Usage: java application module_name\n");
}
}
}
That’s pretty painless, we can compile it now…
~/src $ javac com/michaelclarkeblog/dynamic/application/Application.java
~/src $
Now all that is left it to make our modules. I’m just going to write two, but you can write as many as you want.
package com.michaelclarkeblog.dynamic.modules;
import com.michaelclarkeblog.dynamic.api.*;
public class Mike implements Module {
public String getText() {
return "Hello from Mike's Module!\n";
}
}
package com.michaelclarkeblog.dynamic.modules;
import com.michaelclarkeblog.dynamic.api.*;
public class Another implements Module {
public String getText() {
return "Hello from Another Module!\n";
}
}
And now we compile our modules - notice that we’re NOT re-compiling the main application.
~/src $ javac com/michaelclarkeblog/dynamic/modules/*.java
~/src $
Time to run our program…
~/src $ java com.michaelclarkeblog.dynamic.application.Application
Usage: java application module_name
~/src $ java com.michaelclarkeblog.dynamic.application.Application Mike
Hello from Mike's Module!
~/src $ java com.michaelclarkeblog.dynamic.application.Application Another
Hello from Another Module!
~/src $
Easy eh?!
I know the above doesn’t look very powerful. However, you can make some really dynamic and customisable applications with this technique where your users can even write their own modules for the application - all they need is the api/*.class files (or depending on how many you have you can jar them up)!
Tomorrow (simply because I am no tired) I’m going to post the same application and modules but written in C# and compiled using Mono under Linux. When I said earlier that it was hard to originally find this information for Java (which it appears not to be anymore) it was even harder to find this information for C#, let alone get it working properly!
Posted in Java, Programming | No Comments »
