April 3rd, 2008 Michael Clarke
It’s been quite some time since my last post. This is down to a number of things - the first is that I’ve been really busy and so haven’t had that much time to write a blog post. The second is, with being so busy with mundane things, I’ve not really had anything worth while blogging about.
However, today I was on the phone with a friend (Anton) and our conversation slowly progressed to Dynamic Class Loading, specifically in Java. This made me remember that when I first heard about this technology (about three or four years ago) there wasn’t that much information available on the internet. So, I decided that I’d write a little (or perhaps not so little) post on how to go about getting Dynamically Loadable Classes in Java - it really is pretty easy.
The first thing you need to define is an interface for your modules. Mine is just going to be something simple that had a getText method to get some random text…
package com.michaelclarkeblog.dynamic.api;
public interface Module {
public String getText();
}
You can now compile your API…
~/src $ javac com/michaelclarkeblog/dynamic/api/Module.java
~/src $
The next thing to do is to write your main application that is going to do the module loading. Mine is going to be a simple main method that accepts a java class name as a run-time argument.
package com.michaelclarkeblog.dynamic.application;
/* We must import the URLClassLoader and also our API. */
import java.net.URLClassLoader;
import com.michaelclarkeblog.dynamic.api.*;
public class Application {
public static void main(String[] args) {
Module module;
if ( args.length > 0 ) {
try {
/* Create a new instance of the module.
* Think of this as:
* Module module = new Module()
* Except we're doing it at runtime 
*/
String path = "com.michaelclarkeblog.dynamic.modules." + args[0];
module = (Module)Class.forName(path).getConstructor().newInstance();
/* Run the modules 'getText()' method defined in the interface. */
System.out.println(module.getText());
} catch (Exception e) {
System.err.println(e);
}
} else {
System.out.println("Usage: java application module_name\n");
}
}
}
That’s pretty painless, we can compile it now…
~/src $ javac com/michaelclarkeblog/dynamic/application/Application.java
~/src $
Now all that is left it to make our modules. I’m just going to write two, but you can write as many as you want.
package com.michaelclarkeblog.dynamic.modules;
import com.michaelclarkeblog.dynamic.api.*;
public class Mike implements Module {
public String getText() {
return "Hello from Mike's Module!\n";
}
}
package com.michaelclarkeblog.dynamic.modules;
import com.michaelclarkeblog.dynamic.api.*;
public class Another implements Module {
public String getText() {
return "Hello from Another Module!\n";
}
}
And now we compile our modules - notice that we’re NOT re-compiling the main application.
~/src $ javac com/michaelclarkeblog/dynamic/modules/*.java
~/src $
Time to run our program…
~/src $ java com.michaelclarkeblog.dynamic.application.Application
Usage: java application module_name
~/src $ java com.michaelclarkeblog.dynamic.application.Application Mike
Hello from Mike's Module!
~/src $ java com.michaelclarkeblog.dynamic.application.Application Another
Hello from Another Module!
~/src $
Easy eh?!
I know the above doesn’t look very powerful. However, you can make some really dynamic and customisable applications with this technique where your users can even write their own modules for the application - all they need is the api/*.class files (or depending on how many you have you can jar them up)!
Tomorrow (simply because I am no tired) I’m going to post the same application and modules but written in C# and compiled using Mono under Linux. When I said earlier that it was hard to originally find this information for Java (which it appears not to be anymore) it was even harder to find this information for C#, let alone get it working properly!
Posted in Java, Programming | No Comments »
March 12th, 2008 Michael Clarke
This week I’ve been on the Solaris Internals course at work and I’ve already learnt an awful lot about how Solaris deals with processes (including threads and light weight processes), system calls, scheduling, memory management (hardware and software), paging and swapping! All of this and it’s only 3 days in.
However, one of the things about this is I’ve very quickly realised how far I’ve still got to go with MouthOS and how much I’ve still got to learn. A great example of this is in regards to memory management… Intel, it would appear, offer two different methods of dealing with memory - segmentation and paging. It was my impression that you could basically choose which you wanted to use when writing your OS and so I decided to use segmentation for MouthOS initially.
Unfortunately the Solaris Internals course has given me a bit of a rude awaking - I’ve realised that in order to use Paging you must first enable Segmentation. However, there are many different ways you can section off your RAM before you even get to paging….
Flat Model (which is what I’m going to be changing to) is where you basically say all of the address space (i.e. 32Bit address space is 4GBytes, even if you system only has 2GBytes) can be used for anything with no restrictions on who can do what - i.e. there is no hardware translation of memory accesses in regards to segments. When I say ‘no translation’ that’s not strictly true… because you still have to specific a segment in the GDT the processor will basically prefix every memory address with 0, so the effect is that there is no ‘translation’ but at the same time there is!
There is obviously a problem with this - if I have a process running, and paging isn’t enabled, and I try and access more than 2GBytes of memory from my process it won’t actually physically exist. However, because there are no checks - no exceptions will be raised and your system is likely to fall over pretty quickly. If you’ve got paging going on then there is no problem, a page fault will be generated and the kernel can load in the appropriate page somewhere else.
Protected Flat Model (which is what Solaris, Linux etc use) is where you limit the segment size to the size of the Physical Memory. In other words, instead of providing your processes with the fully available address space, you provide them with an address space that matches the amount of physical RAM. As such, if anything tries to access a memory address greater than the physically available memory a general protection exception will fire and the kernel will be able to deal with it.
Generally when using the Protected Flat Model you would have 4 sections. 2 of these sections will be in Ring 0 and two will be in Ring 3. Ring 0 is the Kernel ring and the two sections will define the text and data sections. It will start at memory location 0 and proceed to the maximum memory location available in physical memory. I.e. 0 -> 2MBytes. The LDT in Ring 3 (user space) will start at 0 + size of kernel and will end at the maximum memory location available in memory.
For example, assuming the Kernel is 2MBytes and the available RAM is 2GBytes, the LDTs would be along the following lines:
Section Ring Protection Base Offset
TEXT 0 r-x 0 2GBytes
DATA 0 rwx 0 2GBytes
TEXT 3 r-x 2MBytes 2GBytes
DATA 3 rwx 2MBytes 2GBytes
This prevents anything which runs in Ring 3 from ever touching the first 2MBytes of memory where the Kernel is stored, however, the Kernel running in Ring 0 can access its own structures and text, and that of Ring 3 also.
Multi-Segment Model basically uses a similar principle to the protected flat model except it adds yet more sections, this time as many as you could care to imagine - each one defining text, data, bss, stacks, heap sections which are available for individual processes and even thread within a process.
Once you’ve sorted out your segmentation, reading my ‘Intel 64 and IA-32 Architectures Software Developer’s Manual’, enabling paging seems to be relatively simple. From what I can tell I need to setup a page directory, pass the memory location of this page directory to cr3 (control register 3 - which actually acts as a context register) and then set bit31 of cr0 (control register 0) to 1… dah da! Oh, and then of course I’ll have to sort out all the other entertaining things such as a page_fault() handler for when we get a request for a page that isn’t currently in memory - however, for the moment I think I’ve got quite enough to be getting on with.
So far my beautiful “proc_enable_paging” assembly routine is looking a little bare….
proc_enable_paging:
mov eax, cr0
and eax, 0x80000000 ; Set bit 31 (PE) = 1.
mov cr0, eax
ret
Posted in MouthOS, Operating Systems | No Comments »
March 10th, 2008 Michael Clarke
I was reading my managers blog this morning and he has done a short post on his visit to the Shell sponsored Wildlife Photographer of the year Exhibition. I was looking at some of the pictures that he’s linked to from his visit and I have to admit that I think this one of a herd of Zebras running is pretty amazing so I thought I’d link to it - it’s well worth a view
Posted in Uncategorized | No Comments »
March 10th, 2008 Michael Clarke
This morning I am about to go on the week long Solaris 10 Internals course at Sun. This is a course that I’ve been looking forward to for some time as it goes into detail about the implementation of the Solaris 10 Kernel - including things such as memory management, file systems, multi-tasking support etc. This started me thinking how great working for Sun really is…
- Everyone at Sun is really friendly and always willing to help and tell you about (and ask for feedback regarding) their own little (or sometimes rather larger) projects.
- The training provided is excellent - so far in a year I’ve been on four courses starting with a mid-range Serengeti course, then the Solaris Intermediate and Solaris Advanced course, and then the high-range Starcat course. I’ve got another two courses to go, the first one being the Solaris Internels and the other one I’ve got to decide on still!
- Sitting where I sit I get prime access to some of the highest level support engineers in Sun - and some of the conversations are immense - but the really great thing is the engineers are always willing to accept suggestions and comments - even from me and my peers - lowly interns.
- Working mostly in the Lab we get access to all of the latest Sun equipment usually before it has been released, and quite often beta versions during the development cycle before the products have even been announced.
- There are always opportunities available - if you want to have a little go at something else for a while you’re encouraged to do so - take for example my recent DTrace work thanks to Paul Humphreys (my manager) and Jon Haslam (one of the DTrace engineers) - it’s not my job, but they’re both being really supportive and offering help and advice - not many jobs you get that sort of support and encouragement…
- You get to travel. Recently I went to our Paris lab in France. Matt has been to Sweeden. Fraser is about to go to Germany and Robin and David are both scheduled to go abroad at some point during the rest of the year - not bad for an internship.
- You really are considered to be a part of the team - not just ‘the students’…. well, we are referred to as ‘the students’ some times, but it doesn’t hold any prejudice… We’re often asked our opinions and suggestions on subjects concerning the lab (and indeed the future of the lab when we’ll have left) and we’re left to (for the most part) manage our workload ourselves - we’re not micro-managed
Thinking about it, I should really have posted this some time ago whilst we were looking for new interns for next year. Unfortunately (or should that be fortunately?) we’ve now filled the three positions for next year - surprisingly all from Aber, so I’m a little bit late… However, I’m sure next years students are going to have a great time in the lab (just like we have).
If you’re interested in working for Sun (as an intern) there are still opportunities available in other departments. I’m not sure of a specific page but if you need/want more info leave a comment and I’ll get you the info… Otherwise, if you’re after a job more permanently then you should visit http://uk.sun.com/aboutsun/employment/
Posted in Solaris, Work | No Comments »
March 2nd, 2008 Michael Clarke
Yesterday I went looking for a new car. After some time browsing Auto Trader I found a truly amazing car. It was a 2.0L P’ Reg Vectra Saloon in Black with Electric Mirrors, Electric Windows, Air Con, CD Multi Changer, Traction Control, Factory Alarm and Immobiliser, Power Steering and only 80,000 miles on the clock. The most amazing thing though was the price - £695!
So, off I went to Flint (about 50 minutes drive from home) to test drive this car. My initial inspection revealed no faults and everything was looking good. The car was in immaculate condition inside and out - could this be too good to be true? Unfortunately the answer to this question was - yes, it was.
After the inspection I got ready for the test drive. At first the car needed to be jump-started because the battery was a little low (fair enough - it has obviously been standing for a while). As I left the garage the car felt a little jerky - initially I suspected this was simply because I wasn’t used to the clutch, however it very quickly became apparent that it was something more serious when after about 2 minutes into the drive the car broke down!
My first thoughts were - oh it’ll just be low on petrol - no, it had almost a quarter of a tank. Perhaps the battery was the problem still. After a few attempts to push start it, it became quite apparent that I needed to call the garage so they could bring their battery pack and get the car going again. So 5 minutes later the guy from the garage arrived… could he get it going? No. I’m still not entirely sure what was wrong with it, but I suspect it’s probably a fault with the immobiliser or the starter motor - either way, I ended up going home without a new car… never mind! It just shows how important a test drive is when buying a car.
So, I’m still on the lookout for a 1.8/2.0 litre Vectra/Mondeo with reasonable mileage if anyone’s got one that they’re selling. Even better would be a MG ZT if anyone has one of them? No? Ah well, pity. For the moment I’m going to borrow one of my parents cars that they’re not using at the moment - a Fiat Marea Weekend, not quite what I was looking for - but never the less a car that is big enough for my needs
Posted in Cars | 2 Comments »
February 27th, 2008 Michael Clarke
Well I’ve not blogged that much recently. However, I’ve been pretty busy and I’ve got quite a lot to blog about. So, I suppose the best place to start would be the middle of last week…
On Wednesday or Thursday last week (I forget exactly) Kim Austin who runs the Campus Ambassador program for Sun informed me that she would (officially now) be taking me on as the Aberystwyth University Campus Ambassador when Chris Talbot leaves at the end of this academic year. I had for some time been a shoe in for the job, however it is now official which is excellent.
At the same time as this Kim also introduced the other interns and myself to the SAI (Sun Academic Initiative) website. This is a excellent website that offers pretty much all of Sun’s courses (except it seems Solaris Internals) for free to students. If you’re a student, specifically a Computer Science (or perhaps Physics/Maths) student, I really recommend you get yourself over to http://www.sun.com/solutions/landing/industry/education/sai/index.xml and register. If your university isn’t already registered they can do, for free! All you need to do is get a professor to register here.
So, what else has happened. Well on Friday we took down our x86 Sunray server, eagain (all our Sunray servers are named after Unix error codes), to replace it with a Huron. ‘Huron’ is the internal code name for the new T5120/T5220 systems Sun sell based on the Niagara 2 chip. These machines are truly amazing - in a 2U rack you get 32 Thread engines (64 total threads) and 64GBs of RAM… This makes the Gnome system monitor a sight to behold…
The funny thing is the way it sort of gives up choosing colours after 4 CPUs…
As most of you will know I’ve recently been given the opportunity to do some work on DTrace, initially just fixing a few bugs and doing some RFEs (Request for Enhancements). Well a number of things have happened since I had my first meeting.
First, Robin (my house mate who thinks that this whole blog is centered round him) has decided to port MouthOS to SPARC. This has proven to be most interesting for both him and myself. Whilst we’ve still not even managed to get characters on the screen we’ve both learned an awful lot about the way that OBP works and how SPARC boots and functions. I have to admit that I’ve been truly amazed at what OBP provides you out of the box compared to x86. OBP defines a number of amazingly useful functions (such as putchar() and printf() etc) so that your boot loader/OS can just call them directly. It also has great debugging utilities built in to the point where you can literally set break points in your OS and then print CPU registers and all sorts of things. All this and it will even give you a nice tree structure with all the hardware devices in the system.
Apart from getting slightly side tracked you may be wondering what this has got to do with DTrace. The simple answer is not much other than this has now meant that Robin will also be doing some Solaris Kernel work - specifically with ‘newboot’ which is designed to allow Solaris (SPARC) to boot ZFS file systems. I’ve no idea when it’s planed to be released or what state the project is at - but I would like to wish Robin good luck with his future Solaris development work, and I also think that another thank you to Paul and also Tim Graves is in order, not to mention Chris Beal who has agreed to be Robins’ mentor.
Anyway, moving on a bit… In regards to my work on DTrace. Apart from endless hours of reading the source code (which I have to admit is the most well commented code I’ve ever seen) yesterday I finally chose my first bug - http://bugs.opensolaris.org/view_bug.do?bug_id=6287021.
It doesn’t look to be a particularly complex or important bug - but it’s a great place to get started with the development as it will require quite a bit of investigation first in order to produce a test case. With a test case I can then start to identify where the error is happening in the code and I can then implement the fix.
I think that pretty much sums up what I’ve been up to recently. This weekend I’m off home again for Mothers Day and my sisters birthday. I’ve booked a couple of days off work (Monday to Wednesday) to have a look around for a new car also, so with any luck I shall hopefully be returning to Farnborough with a car! I’ve seen a couple of Mondeos and a Vectra at a place called Robert Car Sales (about an hour drive from home) - I can’t wait as it’s been so long since I had a car now.
Talking of cars, Matt (another intern at Sun, and friend) had quite a nasty crash the other day on the M1. Some truck driver basically pulled out of the slow lane into the middle lane - into him. His car rolled all the way onto the hard shoulder. Fortunately (bar a cut on his arm) he’s alright - though his car is a write off 
I have given him some advice from when I had my crash so at the very least he’ll know what to expect from the Police, insurance etc. The only problem is the truck driver just drove off and no one seems to have got his number plate!! From the sounds of it he’s going to get off scott free as the CCTV was also down due to road works!
I think that’s everything for now, so until my next exciting installment!
Posted in DTrace, Solaris | No Comments »
February 15th, 2008 Michael Clarke
The other day Matt and I got wind of a decision by workplace resources to replace all of the vending machines at Sun. This was most traumatic news as it meant that today we would have no access to the said vending machines, and as such we would have limited access to coffee!!
In light of this we quickly put operation ‘bring coffee into office’ into effect. I’m sure that we’ve probably got a code name for this operation (no doubt using Java as some kind of pun) however, I can’t remember it - ah well…. Anyway, as such yesterday Matt brought in his coffeemaker and I brought in the coffee… fortunately we both also remembered mugs - phew!
As of yet workplace resources haven’t discovered the coffeemaker - however if they do we’ve got an excellent cover story - it’s a Sun Ultra SPARC Enterprise Java Coffee Maker that we’re having to get part replaced from logistics.
Posted in Random, Work | No Comments »
February 14th, 2008 Michael Clarke
Last night we decided that it’d be a good idea to watch a movie. After an enormous amount of time deciding what to watch we started the process of working out exactly how we were going to watch it. This is always a most interesting experience and usually results in us getting every computer, TV, monitor and media capable device in the living room and then connect everything to everything else wondering why 2 hours have passed and we’ve still not seen the opening titles.
In last nights case, 2 laptops, 2 TVs, 1 set of speakers and 1 XBox later we finally managed to get some kind of output…
In our naivety we’re about to try and watch another movie tonight - who knows what mayhem will ensue…
Posted in Home, Random | No Comments »
February 12th, 2008 Michael Clarke
After my French explorations I booked Friday off and went home for the weekend. I had a nice time back in sunny Wales except for the fact that my family had literally managed to break every single computer in the house - and when I say break, I don’t just mean a few viruses or malfunctions, I literally mean motherboards broken, screens damaged, graphics cards frizzled and who knows whatever else. By the end of the weekend I’d managed to pull the few working parts remaining into two systems - the server (which for the most part was working to begin with except that it wouldn’t boot because it couldn’t find the floppy disk drive) and a computer upstairs for everything else. To be fair I think they’ve had some kind of power surge as one computers PSU fuse had blown along with the fuse in the plug… I think they now plan on buying a few surge protectors :p
When I returned today Paul (my boss) asked for a quick chat in the office. He had some not so great news and some excellent news for me. The first bit of news was that the presentation he and I were going to do at Sheffield university had been taken over by some other guys in the office. This turned out not to be so bad after all as I realised that it was the 29th February anyway - at which point I’m back off home anyway for my sisters 5th birthday. However, the good news (which is very good indeed) is that he has managed to get one of the guys in the Solaris Kernel Engineering department (who works with DTrace) to mentor me (after showing him some of the stuff I’d been doing with MouthOS) for my last two years at university. During this time I will be doing things like fixing DTrace bugs via the Open Solaris community - opening some real possibilities for when I finish to come back and work for Sun in their Solaris engineering department! I’ve got a meeting at 11am tomorrow morning with Jonathan, my mentor to be, which I’m really looking forward to as I’m raring to go and I really can’t wait now until the 10th March when I will be going on the Solaris Internals course - thanks Paul
So, what else has happened over the last few days? On Thursday I got another phone call from ‘Private’ offering me a new phone. I keep telling these people that “I’m not interested and that I’ll deal with Orange directly thank you very much.” However, this prompted me to actually ring Orange and see if I really was entitled to a new phone. The answer was no, not really, but we’ll give you one anyway. Apparently these companies will ring you up, offer you a contract and then (only then) contact Orange to see if it’s going to be alright! The cheek!
Whilst I was on the phone I did say I wanted the Nokia N95. However, they wanted £89 for it. Instead I said I’d have the LG Shine (http://www.lgbloggers.com/)… this was fine except it was out of stock. In the end I decided on the Nokia 6500 Slide (in black which is apparently a Orange exclusive). So far I have been most impressed with the phone. However, as with all new gadgets it’s a requirement to test out every feature it has to offer, one of these features being the camera. I decided that it would be a good idea to take a couple of pictures of the MouthOS development centre (i.e. my desk)…
I have to admit that whilst I would love to be able to say that all of those books are mine, only the top C book and the bottom three Intel books are truly mine - the others are on loan from my house mate and friend Robin. However, I highly suspect that I will be buying my own copies before I have to give them back to him as they have been really useful for ideas and all that malarkey during the development of MouthOS so far - I think another thank you is in order - thanks Robin.
Posted in Family, General, MouthOS, Operating Systems, Solaris, Sun Microsystems, Work | No Comments »
February 7th, 2008 Michael Clarke
I’ve just got back today after a week in France working in the Paris office of Sun. I decided that it would be a very good idea to get up nice and early last Wednesday (5am to be precise) so that there would be no chance of missing my incredibly cheap, no refund, no modifications, return Eurostar journey at 10.30am. This proved to be more than sufficient as I arrived at St. Pancres at 8.30 with 2 hours to spare!
When I arrived in France (about 1.45pm) it then took me a further 2 hours to find my hotel. This was most unfortunate given that my hotel was quite literally 15 minutes walk away from the Eiffel Tower! Ah well. Once I’d finally found my hotel I went for a brief walk around the Eiffel Tower and towards the office - which I found with much more ease.
At the weekend a couple of Friends from University came over to visit. One of them (Ash) had a newspaper cutting which described “how to do the top 5 sights in Paris in one day”. I suspect that we were meant to do this via the Metro, however he insisted that it was a walking exercises. This meant about a 10 - 15 mile walk starting (and then finishing at) Gare du Nord. First we walked to the Pompidou Centre (which is one of the most bizzar places I have ever seen). Then from there we walked to Cathedral Notre dame. From there we walked to the Louvre, then onto the Arc de Triomphe and finally the Eiffel Tower. From there we walked back to Gare du Nord via the major shopping street in Paris. My friends thought this was all very exciting, unfortunately my feet were not too excited by this point having developed numerous blisters!
On Sunday, Monday and Tuesday I was in the office again working on getting the lab back up and running after the power work was completed. There were a few minor problems with the work completed such as the fact that the electrical engineers had used armored cable and not steel wire armored cable (tut tut tut) and the fact that they decided to use some of the 16AMP commandos off some 32AMP to 16AMP splitters we had in the lab because they ran out (go figure, they’ve had the numbers they’ve needed for over 3 months now!). However, on the whole the work went reasonably well and the three power phases are now much more balanced in terms of load.
The last couple of nights in France I went out with Paul (my boss) and JD (one of the old students who is looking after the French lab this year). This was nice, except that on the way back to his Hotel, Paul managed to walk into a glass door and now has quite a bad cut on his nose He said that the manager of the hotel was more interested in cleaning up the blood off the floor than helping him with a plaster.
So, here I am back in the UK and what awaits me in the office? The usual? Well, in reality I suppose the answer is yes, in so far as there is nothing ‘usual’ about our office… and so the usual is the ‘unusual’ most of the time….. Tim Uglow decided to bring his two Samoyeds’ into the office… Thanks to Matt for letting me use his photo….
Posted in General, Random, Sun Microsystems, Work | No Comments »
