Testing the blog waters

Hi. I have been asked to pick up this blog and do a weekly report on OpenSimulator. This is my first test posting as I see if I can actually drive wordpress.

I’m one of the Core Developers and the OSGrid Director and have been around since about r500 in the Spring of 2007. Many of those on IRC know me as “ckrinke”. Those on OSGrid know me as “Charles Krinkeb”. My real-life name is “Charles Krinke”.

OpenSim Docs, OpenSim Nightly Builds

This is just a quick post to let everyone know we’ve added two new resources to the OpenSimulator.org website. The first, we’ve added Doxygen programming docs covering the entire OpenSim codebase – these will be refreshed periodically with every major release that we do. You can find the docs at http://docs.opensimulator.org/

The second major thing we’ve done lately is introduce a Continuous Integration (CI) server which produces regular builds and monitors for build failures. Right now it is configured to produce a new build after every commit – the result of which will be copied to our new builds repository at http://builds.opensimulator.org/

Enjoy – and thanks to everyone involved who helped get this setup!


SVN Module – or, DIY Rollbacks and Backups.

So, earlier this week I committed a prototype of something I have been musing on for DeepThink internally into the OpenSim SVN. It’s a shared region module which acts sort of as a non-realtime datastore. The purpose of which is to allow you to store (and load) region backups to a SVN server.

The SVNModule will use the also-new ‘export’ module, to serialise your region to a collection of XML and binary files that make up your region (XML for anything human editable, ie – object data. Binary for non-human editable components such as terrain data)

It’s worth noting you can use “load-xml2″ on the objects.xml file, and “terrain load heightmap.32″ to import those parts back into the region from outside the SVN server.

Before I continue, I should point out that this is a very untested module – I have only confirmed it works under Windows 32bit, and has not gone under any sort of sufficient testing for large scale use, yet. Proceed at your own risk.

To setup the module, copy the [SVN] section from OpenSim.ini.example to your OpenSim.ini – make sure ‘enabled = true’ is set, and the other details are in there. At the moment it will only work with svn:// and http[s]:// repositories. I am working on SSL/SSH wrapped repositories at the moment, but it is more difficult to implement, so may take some time. You can use any SVN URL (it doesnt have to be the root of a SVN repository). I recommend that you use one subdirectory per server unless you have a high speed link between all your servers — this is because if all servers share a directory, checkout times will include updates from every directory (something on my agenda to implement a workaround).

Start your sim up, and check to make sure you get [SVNBACKUP]: messages on your console, as long as no errors are thrown (of type SvnException) you should be good. If you see an error – there is help text buried in the exception.

Presuming all has gone well, there are several console commands worth reviewing:

svn save – saves a current copy of all your regions to SVN.
svn load – loads SVN head and imports it to your region.
svn load <revision> – loads a specified revision
svn load-region <name> – loads a specified region at head revision
svn load-region <name> <revision> – loads a specified region from a specified SVN revision.

Automatic backups (periodic saves) and several other features are on the cards, but have not yet been implemented. Ideally I would like to convert this to a datastore-style adapter that imports changes live (so you can rollback individual changes), but that may be unwieldy. Some thought is required on the matter.


OpenSim Weekly News – week ending Saturday 12th April 2008

Hello there. For the past couple of weeks I’ve posted a weekly OpenSim development summary to this group blog. However, I’ve realized that I’m a shameless traffic whore (and it does take a surprising amount of time to write the post!), so today’s summary and future editions will be posted to my own blog here. Corrections and comments are still welcome.

OpenSim Weekly News – week ending Saturday 5th April 2008

This week in OpenSim…

  • Teravus fixed a bug which caused problems with region interoperability between Windows and Linux servers. Hurrah for platform neutrality!
  • Adam made more changes to terrain commands.
  • I made a change where the “Take Copy” option in the pie menu really does take a copy of the object rather than the object itself.
  • I made a change so that prims copied using the shift copy method now properly retain their script contents.
  • Teravus updated our embedded OpenDynamicsEngine library. This may have an impact on physics stability when using this engine.
  • sdague reorganized the database namespaces to be OpenSim.Data.* rather than OpenSim.Framework.Data.*. If you’re having trouble running OpenSim, try changing all your references in OpenSim.ini and all the bin/*.xml configuration files (if you are running a grid) to point to the new namespace.
  • I made a change so that the scripts contained in all linked prims can now be properly edited using the ‘Edit linked parts’ option.
  • We had a small hiccup with inventory as the underlying mechanism has changed to CAPS in the latest Linden client. But with lots of preparatory work by MW, Teravus and a very small fix by myself, inventory appears to be working again in the latest client! However, we do still appear to have all the inventory bugs which existed before the client :-)
  • Teravus changed the default packet output queue throttle settings. This should help any problems people have been seeing with prims which appear to be missing when they login (but can still be bumped into!). It should also help with land block problems.

New Terrain Commands

Some time ago, I reworked the terrain engine into a new shiny module. This module is now nearing completion as far as core features go – from here on in, new features will be towards things like v2.0 goals. Since it is now working correctly for most things, I have reworked the console commands back in – with some improvements. Many of you noticed in the last two-three weeks the old commands were ‘depreciated’ – these should now be working again as of r4039/4040.

So, a little guide.

First – I’ve created a new ‘command manager’ which also handles console commands, the goal of here is to enable LSL/Scripting functions for every console command, as well as allowing proper syntax-checked console input plus help. The short of this is you can now type “<module> <command> help” for help about a particular command, or “<module> help” for a list of valid commands. At the moment only “terrain” is supported for modules.

Region# :
terrain help
17:48:06 - ===Terrain===
17:48:06 - * load - Loads a terrain from a specified file.
17:48:06 - * load-tile - Loads a terrain from a section of a larger file.
17:48:06 - * save - Saves the current heightmap to a specified file.
17:48:06 - * fill - Fills the current heightmap with a specified value.
17:48:06 - * newbrushes - Enables experimental brushes which replace the standar
d terrain brushes. WARNING: This is a debug setting and may be removed at any ti
Region# :

You can see the list of commands there, with a brief description of what they do. You can see more detailed help, IE.

Region# :
terrain load-tile help
17:50:26 - == load-tile ==
17:50:26 - Loads a terrain from a section of a larger file.
17:50:26 - = Parameters =
17:50:26 - * filename (String)
17:50:26 -      The file you wish to load from, the file extension determines th
e loader to be used. Supported extensions include:  .r32 (RAW32) .f32 (RAW32) .t
er (Terragen) .raw (LL/SL RAW) .jpg (JPEG) .jpeg (JPEG)
17:50:26 - * file width (Integer)
17:50:26 -      The width of the file in tiles
17:50:26 - * file height (Integer)
17:50:26 -      The height of the file in tiles
17:50:26 - * minimum X tile (Integer)
17:50:26 -      The X region coordinate of the first section on the file
17:50:26 - * minimum Y tile (Integer)
17:50:26 -      The Y region coordinate of the first section on the file
Region# :

You will notice some of the commands have subtly changed since the previous version, load-tile now takes a slightly different input format, measured in tiles rather than pixels. Tiles are assumed to be equal to Constants.RegionSize (256) squared. So, the old format of ‘load-tile file.raw 512 512 1999 1999′ now takes the syntax ‘load-tile file.raw 2 2 1999 1999′.

The help in this new version should be more comprehensive and user-friendly for end users (or I at least hope so), I’m keen to see what users think. Also, if there are any features the previous version had that are not supported now and you were using, please let me know and I will add them to my TODO, but all other things being equal – I’m now going to move on to redesigning the console input for the rest of OpenSim.


Proof of Concept – Advanced Terrain Brushes

In my previous post, I mentioned that I would be working on some new features this weekend, I’ve now finished the first of those – the new terrain brushes. As a proof of concept, I have developed a set of terraforming brushes simulating three different erosion techniques. Read on to learn about the brushes, and how to enable them.

To test these brushes, run the following command on your simulator console – please note that this is permanent until you disable it or restart the simulator.

script terrain newbrushes true

This will swap your default brushes, with the following new and exciting brushes.

  1. Thermal Erosion Brush (Replaces ‘REVERT’ tool) - my favourite. This acts as a sort of a cross between a smooth tool and a flatten tool, it creates very natural looking results if you drag it around a little. The effects of it are sort of similar to what would happen if you blew on a pile of very fine sand from directly above. Spikes and pinnacles get flattened out and their ‘mass’ gets transfered to neighbouring sections, creating a sort of plateau.
  2. Olsen Erosion (Replaces ‘FLATTEN’ tool) – this is a implementation of the oddlabs optimised hybrid erosion routine, described in the paper “Procedural Terrain Generation“. It will flatten flat areas, and sharpen sharp areas – think of it as a light touch you can apply over the top of a terrain to give it more contrast.
  3. Hydraulic Erosion (Replaces ‘SMOOTH’ tool) – this is a fairly powerful brush that needs a little more tweaking. The intended effect of this brush should be like pouring a bottle of water on a sandcastle, in that it will create a river system and drag sediment along with it as it flows.

I’m now going to look at implementing terrain effect filters that apply to an entire sim. Tom Coulthard, author of the GPL’d CAESAR package has been kind enough to give me permission to take a look over his code, and reimplement some of the high-end algorithms inside it for the purpose of developing OpenSim. (Thanks!)

Continuing with Terrain

So, the majority of the internal work of Terrain Module is now done – it works for all the standard operations you expect in SL. We now have the standard array of Raise, Lower, Smooth, Flatten, Noise and Revert brushes, both Paint and Area. The Noise brush still needs a little work as it uses random noise, rather than perlin noise for modifying the terrain, but the rest should behave normally. Parcel support is pending a rewrite of the parcel engine coming soonish, but is on the agenda.

This weekend, I am to knock out at least one, hopefully more of the following features.

  1. Erosion Brushes to complement the raise/lower/smooth/flatten/noise/revert brushes. Thermal erosion I should be able to do in under an hour. Hydraulic erosion is a bit more complex and might probably take longer. My own personal Aerobic erosion routine will probably make a return visit too once I clean it up a little.
  2. Effect Brushes. Part of the original libTerrain I wrote for OpenSim was a very rich and diverse set of routines for doing special effects to a terrain. The original erosion routines sat here, as did a mountain of procedural generators, and things to that effect. I’d like to bring these back, but make them more accessible to end users, I may see if I can instead of hiding them on the console, expose them to the inworld scripting language, I’ll try hunt Tedd to find out how we can do that if we can.
  3. Finish file saving and loading for a variety of file formats. One of the new features I brought into the new Terrain Module was the ability to write File Loaders as plugins to the Terrain Module itself. I also implemented a terragen loader which we didn’t have before. Presently we support loading from RAW32, LLRAW, Terragen, and saving to RAW32, LLRAW and JPEG. The full set I’d like to support is full read/write from the following formats: RAW32, LLRAW, TER, JPEG, BMP, PNG (16-bit ideal), BT, CSV and DEM. We’ll see how I go.
  4. Implement loading/saving for partial heightmaps (the old ‘load-tile’ method) properly.

That’s my TODO for Terrain at the moment. If anyone has any ideas or suggestions for TerrainModule – now is the time to let me know. Comments on here will do just fine.


Introducing … TerrainModule

Sounds boring, doesnt it?

Well, it probably is. But that doesn’t mean I can’t get excited about it, because it’s replacing our old and trusty warhorse of a terrain engine with something that’s less likely to turn your computer into radioactive slag every time someone decides to smooth an area of your sim.

There’s some important changes to how Terrain is handled internally in OpenSim – or to be more correct, how it isnt. Previous versions of OpenSim have relied on the internal terrain engine – this has been a problem because you could not replace it easily, and it was tied pretty heavily into the core of OpenSim. The new engine promises to be more flexible, as you can still operate the sim without it, it’s also closer to the OpenSim spirit of being modular — you can swap individual brushes and rewrite them without major effort.

Over the next few weeks, I aim to write a few more interesting terraforming brushes (my ultimate goal will be to write a series of erosion simulating brushes) and also rebuild part of how the console commands operated.

What the effect on you today will be

  • The terrain console commands will not work. You can still load and save terrain, but you need to use the new ‘script terrain save <filename>’ and ‘script terrain load <filename>’ operations. The other commands have not been implemented just yet.
  • Terrain operations should be faster. Particularly things which apply ‘Area of Effects’ such as select-drag-apply functions.
  • Smoothing and Flattening should no longer freeze the sim when using a debug build.
  • We will be able to support terraforming individual parcels, by the parcel, using the cutout bitmap in the next few days.

So, for developers – if you want to write your own brushes, it’s pretty simple. Take a look in the “Environment/Modules/Terrain” folder to see the directories “FloodBrushes” and “PaintBrushes” — flood brushes are ones that apply to an area (such as smoothing a selection), paint brushes are ones that you can drag across the terrain (such as raising or lowering normally). Brushes are passed a copy of the sim’s ITerrainChannel, and are expected to modify that channel. Please dont create temporary channels if you can avoid it, as creating and destroying channels is computationally very expensive.

Moving on – console commands. Part of the problem with the old Terrain Engine console commands were, they weren’t exactly the best things to document as they were heavily oriented towards how the engine speaks with the old Channel classes, and not how a user should speak to the engine. When I recreate these, I am thinking more along the lines of a rigid programming API interface, which can be self-documenting (ie min vals, max vals, etc) plus also be programmatically accessible, say for the purpose of remote admin. This is going to take a bit more thinking – but it is on the agenda.

Adam out!

Hello World, New Features and Introducing RealXtend

Hello World

I think by the time I write this I should be the first post on here (assuming no-one else posts something first), in which case – welcome. The goal of this little page is to comment on what we are working on, random technical bits and pieces and other musings from the OpenSimulator developers. This should all be synchronised up on the planet.opensim.us page as well which has a few other blogs included in it as well. If there’s any opensim developers who dont want to setup an individual blog but want to post musings, bug me and I’ll add you to the accounts for this page.

New Features, and Introducing RealXtendRealXtend Viewer with Meshes

One of the more suprising developments of this year was the sudden presence of a group called ‘RealXtend’ – we were all fairly taken aback by the amount of development work they have done and are contributing to the OpenSim project, among their achievements is a new client viewer, based on the open source Second Life viewer that adds a few hotly requested features, such as mesh primitives. It’s going to take us a little while to integrate their new features into our codebase properly, but you will be able to download the sources for their modifications from a branch in our SVN repository over the next few days.

You can see a demo of their improvements and download some precompiled versions of their software off their site at RealXtend.org. On the serverside, RealXtend have implemented some nifty features such as Python scripting support (via IronPython), a new server called the ‘Avatar Server’ which allows you to transport your avatars appearance from one seperate OpenSim to a completely different one without any shared grid infrastructure. Hopefully we can get this combined in with some of the new Inter-sim teleport code Dalien has been hacking together for a truly independent grid system.

I wont put a date on how long it takes us to merge their codebase into OpenSim in a clean manner, because it may take some time – but hopefully we can get it done soon.

First post out!



Get every new post delivered to your Inbox.