New year coming ..

.. and so some news related to L.B.A. Remake and I think it's something good. This last few months were I would said pretty tough. A lot thing changed in this time I changed my job and with it place where I currently live but there are things that never change. One of them is my family other are friends and the last one is that I never stopped doing L. B. A. remake :] 

So once again : project is still alive. But this time I have some proof for that :] 

This may look not too much at first sight but truth is that behind a lot of things changed and changed on better. But better solutions came with price: doing them consumed more than two months of work. This is a lot of time at last for me.

But this time is behind me and from now one there should be only better. In the fact effects of the changes that I mention before are still visible and not everything work the way it should but each day stabilize everything and even make some things work a lot better.

So you are probably wondering what will happen next :] and here small announcement : I plan in next few months prepare playable version of game (You can call it like you want: prealpha or techdemo). It will contain only prison map and it purpose will be collecting support from people about game play, mechanics, stability and performance. Everybody will be able to help in future development of project because there will be open access to this version. 

For me it will be first really big test because I need to finish a lot of things and create some others that still don't exist. In the same time I know that the version of game will be far from finished but this release should help me in making it the way every L.B.A. fan want it to go.

So till next post.

Small surprise

Today when I was looking on my blog control panel I find out something surprising. On Frédérick Raynal website: 

in Little Big Adventure section I found out link to my blog. For some people this probably not mean to much but for me it's really something (I wasn't expecting this). Things like that really help a lot in tough moments of development.

I will also add in this place that project is still alive :] I only don't have any proof for it right now. But as history of blog shows this is not first and probably not last situation like that. In most cases this long time of silence is due to my programmer nature. I try to focus on game development itself rather than trying showing each week stuffs that not necessary are made because of need. Good thing about this is that after break I try to return with new stuffs that should show that this time of waiting was worth it.

So till next post folks.

Tech talks: placement new

So another post under sign of programming. This time: placement new. So lets's look on this part of code :
   
...

if (techData.texturesCount[idx] > 0)
{
    tech->m_textures = new(memoryPtr) STextureSlot[tech->m_texturesCapacity];
    memoryPtr += sizeof(STextureSlot) * tech->m_texturesCapacity;
}
else
{
    tech->m_textures = NULL;
}

...

It's a part of White Rabbit Engine materials creator. It's purpose is simple : allocate memory for material + techniques + textures slots. And use placement new to create objects. Everything so all materials data was put in continuous memory. Of course it's nice but there is one problem : this part of code is wrong.

Everything look nice but placement new for arrays add additional data about array before objects. So

  new(memoryPtr) STextureSlot[tech->m_texturesCapacity]

use more memory than :

  sizeof(STextureSlot) * tech->m_texturesCapacity;

For me this mistake ended long looking for place where I override memory and knowledge about two things:

• placement new for arrays add this additional data. 
• it's really nice to have memory guards. But it's even nicer when you check on deleting if it wasn't override :]

Till next post.

Multithreaded rendering (Part 3)

After "little" break lets return to topic. Last time we ended on system that was already split on two threads. This week we will focus how to change it so it work the way we wanted:

So let's start from analyzing a little more situation on which we ended  in last part :

This model look already pretty good and would be enough in some situation. We can use it in this way :

Place marked as "Sync" mean time in which "Gather Data" wait till rendering finish using rApi render list (buffer contain all information's need to render frame). As we see game can update in meantime of rendering. So everything should work faster than in one thread.

Tough times ...

Really tough times came to project. To be precisely it's developed on really powerful netbook acer Aspire One 522. Right now I don't have access to my normal computer and this is only hardware that I have. But don't worry project didn't stop but only going a little slower. To show how much: on my PC project fully recompile in around 3 minutes on netbook the same thing take more than 20 minutes. The same problem is with loading of game which time also extended few times.

So it's not easy but I use this time to speed up everything so I could normal work. It's going really nicely:

• Right now loading time decreased.
• In the same time performance increased.
• Everything mixed with stability changes which are done every time when there is occasion.

Only problem right now is that game is in a little stagnation but I believe that in few nearest weeks this state should change. 

I think that in next post I will return to multithreaded rendering topic. Right now system in engine is really stable and only things that left to do are some polishing of code. So I can finish this article as somebody who wrote fully working system.

So till next post.

Reply to comment

Today I get really interesting comment to Preview 12 from Renkin Hayato. It so interesting that I will respond to it here.

"What do you think about make a team to construct the game? Remake a great game like LBA alone it's kinda suicide! There are very small details that you can forget and i know, do something like this is very wearing and take a lot of time. I hope the project goes forward, unlike other projects remake that were made just to make stupid videos ten thousand years ago!I'm waiting for that since 2009, and i nearly lost the hope of some news about LBA, that stoped to be mentioned at this time."

Question from begin I will leave on a little later :] But about rest you are right. I really know what it's mean to make some good remake it really isn't easy job. Sometimes I even fell little like masochist because my expectation about results are really high. It's event worse because I work in Professional Game Dev as programmer.

But this fact have also really positive aspects: knowledge of techniques, tools and trick that are really handy in development. That thanks to them project is so far in development but in the same time I know that it is on early stage. Why so? Because technology on which it is created still miss a lot to be fully usable.

Great example could be here this projects you mentioned or many indie games. Most people that know about gamedev know that it is easy to create some initial effects: rendering of models, some animations or base mechanics. At this point some people show really nice stuff with all this shaders, particles, models and things like that. But it's still not game it's only some presentation.

This project take different approach which isn't ease. There is created a lot of technology: editor, tools, plugins which are then use to create game. There is no so much pressure to make something to show i.e. in this months shaders were only changed when there was so bug in them (I use glsl and on some cards some construction works on other don't). But in the same time there was really a lot of changes that should influence performance and speed of development.

Right now there stand really big task in front of project : future development. It came to me when I gave access to code repository for one of my good friend. He wrote something like this: I knew that it was big project but never thought that it would be so huge. And it's not that I want do everything alone. Till now I had few proposition for cooperation on project. In most cases everything ended really quickly mostly because I'm looking for people that have skills or real desire to make this project. Of course I found some (or they found me) but it's not easy task. And I think that this is answer for your question.

And to be clear: This project is open on any kind of proposition of help. If your skill aren't enough I can help develop it with knowledge that I have. I have only one request: Think at last twice if you really want to do it because

•  Time I will spend on all talks about cooperation I could spend on game development.
•  Work with me can be sometimes hard because I want this project to be on high level.
•  I sometimes may not have enough time (I'm busy man).

If you want to do it after reading this then ... wait at last the week. And if after this time you still have will to do it contact with me. I will welcome you in development team. If you don't think you have enough strength for this you still can help in a lot of different ways:

•  Comment this project.
•  Do some artworks or even concepts.
•  Share information about this project.

Everything this really help me a lot. Thanks to them I found new strength to continue this project even if it is like Renkin Hayato named it: "kinda suicide".

Query commands execution

So like always start with problem description:

I have some pool of command represented as enumerator. Each of command can have unique data that size can be different. I wanted to create system that allow me in easy way iterate over them and execute. 

After some time I created this implementation:

    template<ECommands::enum cmd>
    bool execCommandTemp(ECommands::ENUM a_cmd, void* a_data)
    {
        if (a_cmd == cmd)
        {
            SCommand<cmd>::execute((SCommand::SData*)a_data);
            return false;
        }
 
        return execCommandTemp<(ECommands::ENUM)(cmd+1)>(a_cmd, a_data);
    }
 
    template<>
    bool execCommandTemp<ECommands::WRAP>(ECommands::ENUM a_cmd, void* a_data)
    {
        return true;
    } 

    bool execCommand( ECommands::ENUM a_cmd, void* a_data )
    {
        return execCommandTemp<(ECommands::ENUM)0>(a_cmd, a_data);
    }

where SCommand look in example such a way:

    template <> struct SCommand<ECommands::BUILD_SHADERPROGRAM>
    {
        struct SData
        {
            CShaderProgram*    program;
        };
 
        static ECommands::ENUM cmdType() { return ECommands::BUILD_SHADERPROGRAM; }
 
        static void execute( SData* a_data )
        {
            a_data->program->build();
        }
    }; 

So why to bother with templates when you can use simple switch :

• Adding new command is very simple create new enum and it's SCommand implementation.
• If you forget about any function it will return error in meantime of compilation.
• You can create very similar function to i.e return size of SCommand::SData

For some people this may be not enough for me it's really good solution because thanks to it adding new command will be quicker and my implementation will be validated in meantime of compilation.

Multithreaded rendering (Part 2)

So in last post on this topic I described a very general look about what is happening inside threads and show some problems that I found in meantime of prototyping. This time we will look a little more on solution that is use in White Rabbit Engine. And few things here:

• Starting from this post I will change a little convention of naming rendering Api. I will use name OpenGL in situation when problem will be 100% connected to it (i.e. extensions, bugs etc.) for everything else I will use rApi so text be a little more general.
• I don't claim that this solution is something really innovative and I will surprise you with it.
• I still work on this system and there is really big probably that some part of it will somehow change in future.

So after this short intro let's move to main topic. As was mentioned earlier engine slowly switched from single thread solution to multiple. Initial loop looked something like that:

First thing that changed was [IO update] that was moved on separate thread. This step was easy other wasn't. There is of course possibility to move [game update] to other threads and left rendering the way it is on main thread. I didn't decide on this mostly because I hadn't any idea on solution which don't have a lot of objects synchronizations (gameplay <-> rendering). And I wanted to avoid it because it connects with big number of critical sections which aren't help later in debugging of code. 

Inconsistency naming convention

And another more technical post :] Today I had occasion to fix some functions in my rendering code. It was look somehow like this: 

struct GPUBuffer

{

    uint32 openGLBufferId;

    uint32 offset;

    uint32 size;

};

...

class CGraphicsManager

{

    ...

    /** @brief Create geometry buffer. */

    bool createBuffers( GPUBuffer* a_buffer,  uint32 a_sizeInBytes,  uint8* a_ptr,  bool a_dynamic);

    /** @brief Flush memory of geometry buffer. */

   void flushBuffer( const GPUBuffer& a_buffer,  uint32 a_offsetInBytes,  uint32 a_sizeInBytes, const uint8* a_ptr);

        

    /** @brief Destroy geometry buffer. */

    void destroyBuffers( GPUBuffer* a_buffer);

    ...

};

So nothing special but I seen here two inconsistency in code convention. First is that once I use pointers and in other occasions references I didn't thought a lot about this and changed everything on pointer. This was something natural to do for me but here came second problem : const .

Function flushBuffer don't change a_buffer in any way. It only use information that it contain to flush OpenGl constructions. So following most of articles and books that I remember, use of const is correct. But looking on purpose of this function it's somehow unnatural. Because even if a_buffer don't change inside function, object that is represented by it change. So in the end I removed const because I think that this way code is a more intuitive.

It's funny like such'a simple thing can force to so much thinking about code convention.

Multithreaded rendering

This time more technical but I think it's really interesting stuff for some of you :]

More than week ago I created new branch on my git repository containing code for "multithreaded rendering". Problem like this are never easy and in my case I needed to deal with OpenGL. And for this who don't know OpenGL is state machine and because of this isn't parallel friendly. All creation of resources, uploads of data, rendering and releasing of it need to be done on the same thread where context were created.

So you see a lot of restriction which complicate already not easy problem. Personaly I thought about solution of it from some time already but couldn't found any that would fit my needs. In most cases there were problems how nicely fit it into my architecture, how to deal with synchronization and few other stuff. But not so long ago I found at last way to deal with it and of course didn't started to code it

In last few years I learned that if we have any idea and we want to realize it, then there is need to give it some time. Start doing something else, talk with people that can give some tips or comments, search for weakness of solution and if we have time wait even week or two. If after everything this idea is still there there is big possibility that it's really worth work that we will put in it. In other way we would lost time reverting changes or dealing with incomplete creation that will need a lot of tweaks later .

So after month of thinking I decided to begin work. I split task from my main thread on few others and right now my threads look like that:

• Main Thread :
• Upload loaded data to GPU
• Rendering using OpenGL api.
• IO thread :
• Process inputs from devices  (mouse, keyboard, pads, oculus rift).
• Game Thread :
• Physics update
• Process game logic
• Prepare scene to rendering
• Loading data to memory (from i.e. disk)
• Working threads
• Execution of jobs

Everything proceed with replacing step by step elements of current pipeline with new one. I done it that way so I could check if any change don't brake anything. In meantime I needed to deal with some problems about which I though a lot ( i.e. how exchange information about what to render between threads ) and this which I didn't thought so much ( i.e. synchronizations between loading of data to OpenGL and using it in rendering ). 

Right now it's really hard to tell how big performance gain will be from this changes but I can say for sure that there is a lot of problems like crashes and bugs that I still need to deal with. But as John Carmack wrote on his twitter: 

"Many types of performance optimization screw up your codebase, but making functions pure for parallel execute actually improves it."

I can say that after all this changes my code starting to be a lot cleaner, stable and I could even say a lot easier to understand.

Till the next post.

A little break

As title say a little brief what was happening in this last few weeks. First part was really nice I took two week of break for vacation and was traveling a lot. Thanks to that I meet interesting peoples and I believe that in future there will be a loot of occasion to meet again. Of course this was only begin because after that I started to finalizing some big changes that will happen for me in near future. I will write about them them at some other occasion.

After this short and boring part lets move on some more interesting topics: what changed  :] 

***

I started work on some few bigger elements. Not all of them were directly connected with game but I would call them "made for fun" i.e: 

LBA Remake on Oculus Rift

It was funny experience but in the same time it pushed my tech a little forward. Mostly because it need this specific way of rendering and also oculus is optional device of multiple purpose (it control parameters of 3D rendering and in the same time is some kind of controller). So multiple system needed to acquire data from it and because of that I needed to change a little bit architecture of engine. On end I get pretty clear and straightforward way of supporting such a devices. 

Of course on oculus rift my work didn't ended. I was doing a lot of cleanup in my old code. It wasn't easy to start this task but it was needed for future growth of project. So I done cleanup, moved everything on unified writing convention, added comments and descriptions how each of cleaned system work. Simple task but took me a lot of time.

When you do a lot of technical stuff you cannot forgot that you do game. Following this believe I added some new character movements. Some were nice and some were put away because I don't have any idea how to create them in the way they wouldn't look stupid i.e. animation after picking important item when twinsen waving his hand up and down. I will probably return to it on some point but if it won't look nice I will probably need to give up on implementing it.

On end I left standard element : bug tracking. In this occasion I done more gameplay test than normal and my list todo grown long in few hours. Some of points are already done but there is still a lot before me. 

Till the next post.

Gui editor

Last two weeks I had pretty busy so I hadn't too much time on working on project. Of course there was few days when I found time then I continued work on prototype of Gui Editor.

From point of looking of final users it could be called useless because most functions don't work and this which works are in really raw state. But from my point of view there was big progress because I think that at last I found direction in which I want progress with it. And this mean a lot for me because till now I didn't had any concept how editor will work and how I want work with it. Of course I always try to make things so they could be later extend without bigger problems so here was another challenge. Right now I think that I got some raw concept that will evolve in another nice tool and only time will show if I'm not mistaken.

Till the next post.

S2d return ...

Here come another info from progresses. I spend a little more time in this weeks on development. One of the reason is weather (it's really hot in Poland) second is that I played and finished new Tomb Raider. But don't worry everything is still going forward. 

And I still do some fixes and changes in technology. A lot of them is could be put in box with label "re-factoring of s2d". S2d was original name of my engine (there still could be find some places in code where I have it in comments)and I try to do some cleanup in code that was write around that time. It's not nice job but must be done earlier or later. I prefer do it now because I will not have to think about later.

Other thing is I returned to making some assets. Below you can find some early wip of new location. Right now it's mostly placeholders that I use to check size and feel of location.

So that's all for this post till the next one.

Lost in code

Ok, this time two weeks breaks so a little longer than normal. But somehow I were lost in my work and forgot to write some post last week :] Yesterday I finished doing initial implementation of memory manager. Thanks to it engine have another powerful tool to help me in development.

Why I moved for this two week on some more engine related stuff?

Because good engine is not only rendering, input, audio like some people may think. Of course this is enough to do some small game but in case of bigger projects things start to be more complicated and because of that we need a lot more technical stuff that make our lives as developer easier. It's influence may not be so visible in game as new full screen effect like Motion Blur or Depth Of Field but they are still there. They help us tracing our bugs, making development easier and in the same quicker. Of course creating them take a lot of time so some people don't decide on creating them but in bigger perspective it's almost never lost of time.

In case of my memory manager my effort resulted in I founding some bugs that would probably take a lot more time to trace it other way if I would knew about them. Other thing is that tool opened before me new paths which leads me to extend this system in even more useful tool.

So right now I can return to doing some more game related stuff.


Till the next week.

Source code experience

Not so long ago I was though about how the code gains its own "experience" over time, You could also refer to this process as „leveling up” mechanics featured in most of RPG games. At first this may sound irrational , but I believe that it's a very common phenomenon and most developers, experienced it at last once during their programming adventure (of course I cannot be sure).

It is possible to go even further, and compare code writing to a RPG game character building and its developement. Your code when you design it is simple, have functionality that you know that are need or you think that they will be need. It's is like creating characters in some RPG games you decide on gender, class, look and some basics parameters. In this point we programmers decide about path that we want to follow with this piece of code and this decision will influence everything later.

Of course hardly ever first decisions are 100% correct. With gaining experience we learn how avoid bad choices but still there are always some mistakes and to overcome them we start to do change in initial plans. In game we would tweak characters so they could pass next enemies. Code is exactly the same you always need to do modifications and cleanups so it be more usable, stable and things like that. Of course you can throw it away and start from scratches but you will loose all this small elements, functions that only wraps some simple operation, error validation and things like that. I called them in this post "source experience" you can call it whatever you want.

They make your code reliable, you know that they not only work but work correct. I don't talk here about hacks because if you have them in your code that mean you cannot relies on it to much. And earlier or later they will remind about themselves.

But what about "Level Up" ? I had such'a situation whit my file system which in some point of development was so stable that I almost didn't do any change in it. But on some point I decide that it could be nicer and do some wrappers that make it's usage easier (your "Level Up"). After that came again time of making small tweaks that make this code really usable.

And I see this pattern often. You code grow with time ( isn't something bad if you do re-factoring ) and your experience. At some point you will catch that you don't touch some things and use them as something natural. Everything because you know that you have there everything you need.

Till next week.

Story about cold coffee ...

Another week of hard work behind me. As you can seem, there are done some changes in blog and a lot of others changes planned. Of course I didn't forgot about development of game. Right now I thinking about direction that I should choose. It's important moment because it will decide about next few months. So I decide to mostly focus on bug fixing and finishing topics that are on my "to do" lists that lie next to my cold coffee.

And it's pretty funny to look at it :] not only because there is a lot of finished tasks. But because it's story about how I create my game. Some pages have some coffee mark because I sometimes put my cup on it. On others you can find some quick notes, calculations and weird pictures that describe ideas that came to me in meantime of doing somethings. As you can see some my notes are maybe a little chaotic but this just like myself and that's why I like them so much.

Till next week.

Preview 13

After so long time here it come LBA preview 13. So as you can see I try very hard to keep my promise about more frequently updates. But what Is going on:

LBA was added to "indie crashe" so if you want this project to get more attention please vote on it here:     

http://indiescrashe3.com/nomination/7A9D3CD9-B726-B9B7-FE33-8FA91ED8A496

There is not too much time but I gotten info that LBA remake participating there very late.

In game itself I do a lot of work on AI behaviors and try create this fully playable prison with all the details and events. There is still a lot to do but it's worth it mostly because I think about releasing something like "alpha techdemo" where you will be able to play this first location and give me some feedback what could be done better or what will not work.

Other thing that I done this week were work on my pipeline to create assets. I done a lot of improvement in this mater one of it was plugin "verbose" mode other tweaks to exporter so time to create and export mesh to game was as short as possible. And of course there was a lot of small fixes and changes in editor so everything was even more intuitive and stable.

And that's all in this week, have a nice watching and till next week.

Pirate Crystal's released

I contacted with people from Pirate Cristal's team (with this that I had contact to) and after talk we decided to release game. I'm not 100% sure if this is the final version but it's only I have on my disk so there is no big choice. I think that game is pretty polished but there is small exception:

English translation is done today thanks to google translator. I didn't even read them only do them quick and recover installer script (it's still taken me few hours). So text in English version can be weird and to tell truth I'm not feel enough confident in my linguistic skills to do it correct. So if anybody want help and do some normal translation PL->EN or do correction of English version I'm open on proposition and can help with explanation how to do it.

So after this short explanation I present you: Pirate Cristal's Download

Progress update

This time still without any movie but not without anything :] In this moment I spend a lot of time on development so sorry for this long silence. So short what is happened with game in this long time:

• Animation system changed so it's use was a lot easier in use and effects were better. So moment of playing sounds of steps, action on hits and shooting is a lot better synchronized with animations. 
• Done support of 3D sounds so everything started to live.
• Added multiple lights support.
• Nice water effect.
• Worked on AI for enemies and they behaviors.
• Falling from heights have nice effects.
• Game support multiple CPU (Moved on multithreaded code).
• A lot of fixes so everything was stable and work nice.

Thanks to GIT history I can say that from last post about development in 29 november 2012 I done:

• Uploaded 144 revision with only code (I often upload change list with multiple things in them).
• Modified 7577 files.
• Changed 121890 lines (without files I added and removed with them 820442 lines)

So as you can see I done a lot. Of course not everything is done directly in game a lot changed in technology on which it's created :

I know that for some people developing of own technology is not the best way of do things. For me it's big playground (seriously right now it's really big) where I can do what I want and how I want. So resuming I still don't know if decision of creating own technology was correct but I know that I won't regret it because I learn a lot making it and I will learn a lot in future development.

I will try to upload a lot more stuff so there was no more such a long break in posts.

Funny thing.

It's a funny thing when you do cleanup in your old code and find comments like this:

/** @brief particle emitter (interface) */
class WRE_API CConfigScript
{....}

Of course class CConfigScript is use to parse any kinds of config files not only particles emitter:]