May 16th, 2008 Michael Clarke
It’s taken me some time to gather the enthusiasm to do this - but I’ve finally moved my MouthOS source over to my web server and installed trac. So, you can view the MouthOS wiki and source at:
http://www.michael-clarke-blog.com/trac/MouthOS
The older versions are in /proto. Kernel version 0.0.1, 0.0.2 and 0.0.3 are broken with some pretty big bugs (for such small programs) such as triple faults and Kernel panics.
The most stable version at the moment is version 0.0.03 in /releases.
The stuff in /trunk is the current development with all the new HAL (Hardware Abstraction Layer) stuff. I’m still in the process of porting a lot of the proto-0.0.3 and releases-0.0.3 stuff to /trunk - so this is pretty useless at the moment 
But you’ll get the general idea as to where the project is heading from /trunk in terms of the HAL, Memory Management etc.
At some point I’m also going to add anonymous svn access so you’ll all be able to checkout the source code and have a play - but at least for the moment you can view the code, the wiki etc.
For the moment - enjoy 
Posted in MouthOS, Operating Systems, Programming | No Comments »
May 11th, 2008 Michael Clarke
Yesterday I decided that since the weather was so nice it’d be a good idea to go for a nice bike ride. One of the guys who works in the call center, Vlad, showed interest in a bike ride also having been a bit of a keen cyclist before some woman drove over him!
So, we met at Sun about 3pm with plenty of water and every tool under the sun that we could possible need. The plan was to cycle to Fleet, then around Fleet lake and perhaps pop into The Heron on the Lake for a quick pint before cycling back - no too bad at all.
Things were going excellently - we arrived at fleet and got about 1/2 way around the lake when Vlad pulled up and said - “your back tyre is looking a little low”. No problem we thought - we’ll just pump it up a bit and off we go. No such luck - the inner tube had rotated slightly in the tire resulting in the valve being at such a funny angle that no pump would attach to it. No matter what we tried we couldn’t move the inner tube around so in the end we figured we’d just have to let the whole tyre down, move the inner tube around, and then pump it up again.
We were soon pumping the tyre up again - but every time we stopped there was a loud hissing - puncture. Unfortunately the puncture turned out to be right on the valve - probably because the inner tube had pinched… No puncture repair kit was going to fix that and the only inner tube we had was too big for my tyre.
After some debating I decided the best thing to do was to leave my bike nice and hidden in the bushes. Walk home (about 1 1/2 hours), pick up the car and then drive back to pick up my bike. The only problem with this truly cunning plan was that Vlad and I weren’t entirely sure where we were… all we knew is we were approximately 1/2 way around the lake (though it later turned out to be 3/4 way around the lake - throwing our directions off slightly).
However, all was not lost. Vlad’s mobile had GPS on it and so we were able to walk roughly in the direction of home. At one point we slightly overshot our turning and the distance started increasing so I decided to take a shortcut through some woods rather than following the path. This seemed quite productive until we hit a locked gate next to a road… wondering why the gate onto the woods we’d just walked through was closed we climbed over and then saw the ‘DANGER! Army Training Grounds. DO NOT ENTER” signs in bright red coupled with the ‘Do NOT pickup suspicious objects” with a delightful picture of an explosion - oops.
Anyway… this is basically where the excitement ended. From here on we walked home, got the car, picked up the bike and then decided a BBQ was an excellent idea… the only think I’m regretting about the whole experience is that our chosen location for our attempted repairs on my bike was riddled with midgies - I now have rather red, bumpy and itchy legs I count approximately 16 bites on one leg and 44 on the other… time to go to the super market and pick up some cream for insect bites me thinks…
On a lighter note it turns out that Friday was Fraser’s (my house mate) birthday. He kept that very quiet - the only reason I found out was because I saw one of his cards…. As such we bought a cake which we all consumed after the BBQ last night.
Posted in Play, Random | No Comments »
May 7th, 2008 Michael Clarke
Given how this weekend just passed was a bank holiday weekend I decided to go back home to Sunny Wales. Usually it’s a bit of a joke - ‘Sunny Wales’ - but this weekend it was beautiful weather and so we visited Lake Vyrnwy - about a 15 minute drive from home…
I’ve got to admit that being down south in Farnborough for the last year really makes you appriciate the views back home that you normally just take for granted.
Posted in General | No Comments »
April 18th, 2008 Michael Clarke
How bazaar - today Robin announced that the previous occupant of his desk at work had left a pair of handcuffs in the top draw. What is more, on inspection, it appears he was telling the truth - there really was a pair of handcuffs in his top draw. Hmmm… my mind truly boggles.
Posted in Random | No Comments »
April 13th, 2008 Michael Clarke
I’ve finally made the move. For some time now I’ve been using bash in vi mode - however there is just one little niggle with it, every now and then the display messes up and you start seeing things like this:
le/why/has/this/moved/here?puter ~ $ mplayer /some/long/path/to/some/fi
That is really really really annoying when it happens.
Anyway, a colleague mentioned some time ago that this was the reason he moved back to Korn Shell… so I figured I’d give it a go. So far the only thing I’ve not managed to get working is coloured ls output…. Everything else is working, tab completion (even though I’m trying to use Esc + \ instead), coloured PS1 prompt etc etc … most importantly though, vi mode hasn’t, as of yet, messed up! 
I’m starting to get a little concerned though - I’ve moved away from Gnome to Fluxbox at home and to fvwm2 at work… then on top of this I’ve now changed my shell from bash to ksh - what is happening!?!
Edit: I’ve now got coloured ls working… I was too involved reading the .bashrc files in Gentoo trying to work out how bash does it… instead I should have just typed man ls….
--color[=WHEN]
control whether color is used to distinguish file types. WHEN may be `never',
`always', or `auto'
Duh!
michaelfclarke@mikes-computer michaelfclarke $ alias ls
ls='ls --color=always'
Posted in Linux, Solaris, Unix | 3 Comments »
April 5th, 2008 Michael Clarke
The other day I wrote a small tutorial on getting dynamic class loading working for Java. In that post I mentioned that it could also be done in C# - but that it was more difficult, and from my personal experience, there is less information available about it - so here is how to do it!
First, as with the Java, we need to create our API. However, unlike the Java there is an additional requirement - a ‘module manager’ class needs to be within the API. As such you will need to create these two files…
using System;
using System.Reflection;
namespace com.michaelclarkeblog.api {
public static class ModuleManager {
public static Module getInstance(String fileName) {
/* Load in the assembly. */
Assembly moduleAssembly = Assembly.LoadFile(fileName);
/* Get the types of classes that are in this assembly. */
Type[] types = moduleAssembly.GetTypes();
/* Loop through the types in the assembly until we find
* a class that implements a Module.
*/
foreach (Type type in types) {
if (type.GetInterface("Module") != null) {
/* Create a new instance of the 'Module'. */
return (Module)Activator.CreateInstance(type);
}
}
return null;
}
}
}
using System;
namespace com.michaelclarke.dynamic.api {
public interface Module {
String getText();
}
}
Once you’ve got the two files you can compile them into a library….
api $ mcs -t:library Module.cs ModuleManager.cs
api $ ls -a | grep .dll
Module.dll
api $
Now that we’ve made the API the next thing to write is the application that is going to use it for loading modules. This is a very simple application…
using System;
using com.michaelclarkeblog.dynamic.api;
namespace com.michaelclarkeblog.dynamic.application {
public class Application {
public static void Main(String[] args) {
if (args.Length > 0 ) {
Module module = ModuleManager.getInstance(args[0]);
Console.WriteLine(module.getText());
} else {
Console.WriteLine("Usage: mono Application module_name");
}
}
}
}
Now we can compile our ‘Application’ against the API…
application $ mcs -reference:../api/Module.dll Application.cs
application $ ls -la | grep exe
-rwxr-xr-x 1 michaelfclarke michaelfclarke 3072 Apr 5 19:32 Application.exe
application $
And now finally we can write a couple of modules!
using System;
using com.michaelclarkeblog.dynamic.api;
namespace com.michaelclarkeblog.dynamic.modules {
public class Mike : Module {
public String getText() {
return "Hello from Mike's Module!";
}
}
}
using System;
using com.michaelclarkeblog.dynamic.api;
namespace com.michaelclarkeblog.dynamic.modules {
public class Another : Module {
public String getText() {
return "Hello from Another Module!";
}
}
}
Compile the modules…
modules $ mcs -t:library -reference:../api/Module.dll Another.cs
modules $ mcs -t:library -reference:../api/Module.dll Mike.cs
modules $ ls -la | grep dll
-rwxr-xr-x 1 michaelfclarke michaelfclarke 3072 Apr 5 19:36 Another.dll
-rwxr-xr-x 1 michaelfclarke michaelfclarke 3072 Apr 5 19:36 Mike.dll
modules $
There is just one last thing we need to do before we can run our application. We need to copy the Module.dll API from the api directory into the application directory…
modules $ cd ../application
application $ cp ../api/Module.dll .
And now lets try running our dynamic application…
application $ mono Application.exe ../modules/Mike.dll
Hello from Mike's Module!
application $ mono Application.exe ../modules/Another.dll
Hello from Another Module!
application $ mono Application.exe
Usage: mono Application module_name
Posted in C#, Programming | No Comments »
April 3rd, 2008 Michael Clarke
It’s been quite some time since my last post. This is down to a number of things - the first is that I’ve been really busy and so haven’t had that much time to write a blog post. The second is, with being so busy with mundane things, I’ve not really had anything worth while blogging about.
However, today I was on the phone with a friend (Anton) and our conversation slowly progressed to Dynamic Class Loading, specifically in Java. This made me remember that when I first heard about this technology (about three or four years ago) there wasn’t that much information available on the internet. So, I decided that I’d write a little (or perhaps not so little) post on how to go about getting Dynamically Loadable Classes in Java - it really is pretty easy.
The first thing you need to define is an interface for your modules. Mine is just going to be something simple that had a getText method to get some random text…
package com.michaelclarkeblog.dynamic.api;
public interface Module {
public String getText();
}
You can now compile your API…
~/src $ javac com/michaelclarkeblog/dynamic/api/Module.java
~/src $
The next thing to do is to write your main application that is going to do the module loading. Mine is going to be a simple main method that accepts a java class name as a run-time argument.
package com.michaelclarkeblog.dynamic.application;
/* We must import the URLClassLoader and also our API. */
import java.net.URLClassLoader;
import com.michaelclarkeblog.dynamic.api.*;
public class Application {
public static void main(String[] args) {
Module module;
if ( args.length > 0 ) {
try {
/* Create a new instance of the module.
* Think of this as:
* Module module = new Module()
* Except we're doing it at runtime
*/
String path = "com.michaelclarkeblog.dynamic.modules." + args[0];
module = (Module)Class.forName(path).getConstructor().newInstance();
/* Run the modules 'getText()' method defined in the interface. */
System.out.println(module.getText());
} catch (Exception e) {
System.err.println(e);
}
} else {
System.out.println("Usage: java application module_name\n");
}
}
}
That’s pretty painless, we can compile it now…
~/src $ javac com/michaelclarkeblog/dynamic/application/Application.java
~/src $
Now all that is left it to make our modules. I’m just going to write two, but you can write as many as you want.
package com.michaelclarkeblog.dynamic.modules;
import com.michaelclarkeblog.dynamic.api.*;
public class Mike implements Module {
public String getText() {
return "Hello from Mike's Module!\n";
}
}
package com.michaelclarkeblog.dynamic.modules;
import com.michaelclarkeblog.dynamic.api.*;
public class Another implements Module {
public String getText() {
return "Hello from Another Module!\n";
}
}
And now we compile our modules - notice that we’re NOT re-compiling the main application.
~/src $ javac com/michaelclarkeblog/dynamic/modules/*.java
~/src $
Time to run our program…
~/src $ java com.michaelclarkeblog.dynamic.application.Application
Usage: java application module_name
~/src $ java com.michaelclarkeblog.dynamic.application.Application Mike
Hello from Mike's Module!
~/src $ java com.michaelclarkeblog.dynamic.application.Application Another
Hello from Another Module!
~/src $
Easy eh?!
I know the above doesn’t look very powerful. However, you can make some really dynamic and customisable applications with this technique where your users can even write their own modules for the application - all they need is the api/*.class files (or depending on how many you have you can jar them up)!
Tomorrow (simply because I am no tired) I’m going to post the same application and modules but written in C# and compiled using Mono under Linux. When I said earlier that it was hard to originally find this information for Java (which it appears not to be anymore) it was even harder to find this information for C#, let alone get it working properly!
Posted in Java, Programming | No Comments »
March 12th, 2008 Michael Clarke
This week I’ve been on the Solaris Internals course at work and I’ve already learnt an awful lot about how Solaris deals with processes (including threads and light weight processes), system calls, scheduling, memory management (hardware and software), paging and swapping! All of this and it’s only 3 days in.
However, one of the things about this is I’ve very quickly realised how far I’ve still got to go with MouthOS and how much I’ve still got to learn. A great example of this is in regards to memory management… Intel, it would appear, offer two different methods of dealing with memory - segmentation and paging. It was my impression that you could basically choose which you wanted to use when writing your OS and so I decided to use segmentation for MouthOS initially.
Unfortunately the Solaris Internals course has given me a bit of a rude awaking - I’ve realised that in order to use Paging you must first enable Segmentation. However, there are many different ways you can section off your RAM before you even get to paging….
Flat Model (which is what I’m going to be changing to) is where you basically say all of the address space (i.e. 32Bit address space is 4GBytes, even if you system only has 2GBytes) can be used for anything with no restrictions on who can do what - i.e. there is no hardware translation of memory accesses in regards to segments. When I say ‘no translation’ that’s not strictly true… because you still have to specific a segment in the GDT the processor will basically prefix every memory address with 0, so the effect is that there is no ‘translation’ but at the same time there is!
There is obviously a problem with this - if I have a process running, and paging isn’t enabled, and I try and access more than 2GBytes of memory from my process it won’t actually physically exist. However, because there are no checks - no exceptions will be raised and your system is likely to fall over pretty quickly. If you’ve got paging going on then there is no problem, a page fault will be generated and the kernel can load in the appropriate page somewhere else.
Protected Flat Model (which is what Solaris, Linux etc use) is where you limit the segment size to the size of the Physical Memory. In other words, instead of providing your processes with the fully available address space, you provide them with an address space that matches the amount of physical RAM. As such, if anything tries to access a memory address greater than the physically available memory a general protection exception will fire and the kernel will be able to deal with it.
Generally when using the Protected Flat Model you would have 4 sections. 2 of these sections will be in Ring 0 and two will be in Ring 3. Ring 0 is the Kernel ring and the two sections will define the text and data sections. It will start at memory location 0 and proceed to the maximum memory location available in physical memory. I.e. 0 -> 2MBytes. The LDT in Ring 3 (user space) will start at 0 + size of kernel and will end at the maximum memory location available in memory.
For example, assuming the Kernel is 2MBytes and the available RAM is 2GBytes, the LDTs would be along the following lines:
Section Ring Protection Base Offset
TEXT 0 r-x 0 2GBytes
DATA 0 rwx 0 2GBytes
TEXT 3 r-x 2MBytes 2GBytes
DATA 3 rwx 2MBytes 2GBytes
This prevents anything which runs in Ring 3 from ever touching the first 2MBytes of memory where the Kernel is stored, however, the Kernel running in Ring 0 can access its own structures and text, and that of Ring 3 also.
Multi-Segment Model basically uses a similar principle to the protected flat model except it adds yet more sections, this time as many as you could care to imagine - each one defining text, data, bss, stacks, heap sections which are available for individual processes and even thread within a process.
Once you’ve sorted out your segmentation, reading my ‘Intel 64 and IA-32 Architectures Software Developer’s Manual’, enabling paging seems to be relatively simple. From what I can tell I need to setup a page directory, pass the memory location of this page directory to cr3 (control register 3 - which actually acts as a context register) and then set bit31 of cr0 (control register 0) to 1… dah da! Oh, and then of course I’ll have to sort out all the other entertaining things such as a page_fault() handler for when we get a request for a page that isn’t currently in memory - however, for the moment I think I’ve got quite enough to be getting on with.
So far my beautiful “proc_enable_paging” assembly routine is looking a little bare….
proc_enable_paging:
mov eax, cr0
and eax, 0x80000000 ; Set bit 31 (PE) = 1.
mov cr0, eax
ret
Posted in MouthOS, Operating Systems | No Comments »
March 10th, 2008 Michael Clarke
I was reading my managers blog this morning and he has done a short post on his visit to the Shell sponsored Wildlife Photographer of the year Exhibition. I was looking at some of the pictures that he’s linked to from his visit and I have to admit that I think this one of a herd of Zebras running is pretty amazing so I thought I’d link to it - it’s well worth a view
Posted in Uncategorized | No Comments »
March 10th, 2008 Michael Clarke
This morning I am about to go on the week long Solaris 10 Internals course at Sun. This is a course that I’ve been looking forward to for some time as it goes into detail about the implementation of the Solaris 10 Kernel - including things such as memory management, file systems, multi-tasking support etc. This started me thinking how great working for Sun really is…
- Everyone at Sun is really friendly and always willing to help and tell you about (and ask for feedback regarding) their own little (or sometimes rather larger) projects.
- The training provided is excellent - so far in a year I’ve been on four courses starting with a mid-range Serengeti course, then the Solaris Intermediate and Solaris Advanced course, and then the high-range Starcat course. I’ve got another two courses to go, the first one being the Solaris Internels and the other one I’ve got to decide on still!
- Sitting where I sit I get prime access to some of the highest level support engineers in Sun - and some of the conversations are immense - but the really great thing is the engineers are always willing to accept suggestions and comments - even from me and my peers - lowly interns.
- Working mostly in the Lab we get access to all of the latest Sun equipment usually before it has been released, and quite often beta versions during the development cycle before the products have even been announced.
- There are always opportunities available - if you want to have a little go at something else for a while you’re encouraged to do so - take for example my recent DTrace work thanks to Paul Humphreys (my manager) and Jon Haslam (one of the DTrace engineers) - it’s not my job, but they’re both being really supportive and offering help and advice - not many jobs you get that sort of support and encouragement…
- You get to travel. Recently I went to our Paris lab in France. Matt has been to Sweeden. Fraser is about to go to Germany and Robin and David are both scheduled to go abroad at some point during the rest of the year - not bad for an internship.
- You really are considered to be a part of the team - not just ‘the students’…. well, we are referred to as ‘the students’ some times, but it doesn’t hold any prejudice… We’re often asked our opinions and suggestions on subjects concerning the lab (and indeed the future of the lab when we’ll have left) and we’re left to (for the most part) manage our workload ourselves - we’re not micro-managed
Thinking about it, I should really have posted this some time ago whilst we were looking for new interns for next year. Unfortunately (or should that be fortunately?) we’ve now filled the three positions for next year - surprisingly all from Aber, so I’m a little bit late… However, I’m sure next years students are going to have a great time in the lab (just like we have).
If you’re interested in working for Sun (as an intern) there are still opportunities available in other departments. I’m not sure of a specific page but if you need/want more info leave a comment and I’ll get you the info… Otherwise, if you’re after a job more permanently then you should visit http://uk.sun.com/aboutsun/employment/
Posted in Solaris, Work | No Comments »
