Scheduling

It's weird how certain concepts simply stay out of your field of 'conceivability', so to speak, until they suddenly pop in and you feel silly for not considering them earlier.

Setting up a schedule for myself has been such a concept. I have read about the concept and its advantages several times before, but for some reason I have just shrugged and never considered it seriously. And I don't really know why - that's the paradox of gestalt shifts - once you have shifted, you're unable to see the reasoning behind your old view (unless you have written it down, or something like that).

I believe that perhaps part of the reason I have been reluctant to set up a schedule is my slightly irregular sleeping habits. I have thought it more important to be rested than to wake up at a certain time. And I still do - working ten hours at sixty percent is worse than working eight at ninety. And my brain is really sensitive to this. It's like sleeping badly puts some kind of insulator between the synapses so they're unable to fire properly.

However, there are a couple of reasons I presently have for willing to try out a schedule nonetheless:

If it turns out that I'm unable to function properly because I am determined to wake up at a certain time, I could always wait with setting up the schedule until morning the same day. That way, I know how much time I have for disposal.

However, I presently have another theory: That my irregular sleep is in part due to my not having any obligations to get up in the morning. Currently, I have a research position, which means I can pretty much come and go as I want. Could this have a negative effect? Perhaps if I approach it more like I would a regular job, my brain somehow would get more 'incentive' to sleep properly during the night? You see, my problem isn't that I cannot fall asleep in the evening - I usually do pretty quickly. Rather, the problem is that my sleep is light and not 'restful' enough. Also, I usually wake up before time, and if I get up at that time, I will be tired.

In other words, this is going to be an experiment. I will schedule the following day the night before, including a time at which I wake up and a time at which I go to bed, and everything in between. Naturally, it will be impossible to follow such a schedule to the point - unexpected events do occur, of course, and there are some tasks which are hard to approximate in terms of time needed for completion. However, those things I believe will come with experience. The first hurdle is actually following through with it.

Game review: Phoenix Wright: Ace Attorney

Despite all my wishes to be a productive person, sometimes I somehow end up playing some computer or video game. Recently I have been playing Phoenix Wright: Ace Attorney, and thought I'd just briefly review it.

First off, I don't like assigning one number to games, since the quality of a game can have many dimensions. SO I'm just going to write what I like about the game and what I don't like.

The premise

The game I played is for Nintendo DS, and it's a kind of point-and-click mystery solving and courtroom game, which to my knowledge is pretty unique in the market. It's animated with semi-moving anime frames. You play as Phoenix Wright, a lawyer straight out of law school, as he takes on his first cases as a defense attorney. The first mission is a simple trial, where you have to pick the witnesses testimonies apart, pressing every point and using evidence to bring to light contradictions in their testimonies. Later, you also play the role of the evidence-gatherer, which you usually do much better than the local police force anyway. There are several clashes with the arch-nemesis, Miles Edgeworth (who has later gotten games of his own). This game is the first in a series of several.

The good

Phoenix Wright (the character) is pretty awesome, although playing the game I got a different impression from what I had from all the internet memes about him.

Y'know - these ones.

He's a bit more insecure than I thought prior to playing. But I like the character, and Miles is also pretty cool, although sometimes I wish the anime industry would find another archetype than the 'brooding dark-haired guy' to be the cool dude.

The trials are hilarious and very entertaining. Whenever you manage to point out a contradiction and cool music starts playing, you feel like being a defense attorney would be the coolest job in the world. There's plenty of humor there, and especially if you're geared towards Japanese-style humor, you'll laugh out loud a lot. I did, at least. Most trials are pretty far-fetched in terms of how they are conducted and what is accepted as evidence and so on, but it's not much worse than your average American lawyer show.
Also, I want to mention the 'effects' as a good point of this game. In trial, when the attorneys are making a point, they are punching their fists on the desk in a really cool way. And whenever something 'unexpected' is happening, the effects really help bring this out by changing the music, kind of shaking the screen and in general putting surprised faces on everyone.

The bad

The evidence collection becomes pretty tedious, especially when you have to move through areas in a very slow manner (i.e. you cannot necessarily move from a given area to the area you want to be in - you have to go through all the 'intermediate' areas first).

The both

The music is really great at times (i.e. during the trials) but at other times it can get a bit jarring (i.e. during evidence collection).

The graphics.. honestly, for a game such as this, realistic graphics is by no means something I want. The graphics do a good job without being extravagant.

In summary

Despite the shortcomings of the game (i.e. the evidence collection phase) I would heartily recommend playing it, simply because they have a unique experience to offer: Being an awesome defense attorney who fights injustice and tears down even the most arrogant of prosecutors. Get your OBJECTION!s on and play it!

Check for duplicates in Python

Today's trick: Check whether a Python container cont contains duplicates!

if len(cont) != len(set(cont)): raise myError

Neat!

Fugitive.vim

All of my posts presently are written in the nick of time. It could be symptomatic of either bad planning or too much to do, but I will try to keep up the current schedule for a while longer at least.

As stated in another post, one of the plugins I am using for vim is the fugitive.vim plugin, written by Tim Pope. Its description is "A Git wrapper so awesome, it should be illegal". And so far, I have to agree - it is pretty darn awesome.

My favorite feature so far is the :Gdiff mode. It has done wonders to the tidyness of my git history. Used to be, I would edit a file, spot a minor bug that was unrelated to whatever I was currently implementing, and then I had to either a)fix the bug and implement whatever, commiting everything in one large chunk, thus messing up the Git history, or b) stash the changes so far, fix the bug, commit it, then continue implementing.

Option b) isn't actually that bad in itself. It just takes a little more time. However, if you spot multiple bugs, or do several separate modifications of the file, the stashing can get a little messy.

Now, the Gdiff command opens up the file you're editing together with the current HEAD version of that file. (Or actually, that's not exactly what is opened, but I have to research more about how Git does stuff before I have more to say). It opens these two files in Diff mode (which I didn't even know about prior to this). It then allows you to choose hunks to stage for a commit, so that you don't have to commit everything in the file at once, if you don't want to. (A hunk is one continuous piece of changed 'stuff'). However, you can even break up the hunks by specifying the lines you want to 'diffput'.

In short - it's awesome. It has other neat features as well, but those will have to come at another time. I might also write a more technical piece on the Gdiff thing.

TRAPs

As a true nerd, I am currently GMing an RPG campaign.

Preparing for sessions can be a chore - I find myself wondering how to structure the stuff I'm making up, and thinking about ways to organize often take more time than actual campaign writing.

However, there is one technique that I am extremely thankful for coming across - the TRAPs method, invented by Ry at Enworld forums: ry's Threats, Rewards, Assets and Problems (TRAPs)

Not that kind.

 
It's really making my life as a GM a whole lot easier, because it's a simple algorithm for fleshing out an adventure or encounter: Everything you add should be either a Threat, Reward, Asset or Problem. If you're introducing something that's neither, it's ineffective. And before you complain about 'atmosphere' and so on - you can easily turn either of these things into stuff that provide atmosphere.

Right now, I don't have enough time to write about this more elaborately (I have to prepare for the session), but once I have tried it out a bit more, I will try to write down my experiences.

Shared library writeup: Part 2

I ended last time talking about how the program header file contains references to the segments of an ELF file. Now, these segments can have various access permissions - some parts are executable but not writable, while some are writable but not executable.

Having a lot of non-writable segments is a good thing, since it means, in addition to data being protected from unintentional or malignant modification, that these segments can be shared if there are several applications that use them.

The way that the kernel knows what kind of segments are of what type is by reading the program header table, where this information is located. This table is represented by C structs called ELF32_Phdr or ELF64_Phdr.

However, the program header table is not located at a fixed place in an ELF file. The only thing that is fixed is the ELF header, which is always put at 'offset' zero, meaning the beginning of the file, essentially. (Offset means how many bytes from the beginning something is located). This header is also represented by a C struct, called ELF32_Ehdr or ELF64_Ehdr (the 32 or 64 refers to whether the computer architecture is 32-bit or 64-bit, respectively - i.e., all its registers, memory addresses and buses have sizes of 32 bits or 64 bits.)

Now, the ELF header struct contains several pieces of information (fields) that are necessary to determine where the program header is. Writing down these pieces means essentially copy-pasting the article I'm reading, so I think I will not go down to that level of granulation.

Once the kernel has found the program header table, it can start reading information about each segment. The first thing it needs to know is which type the segment is, which is represented by the p_type field of the program header table struct. If this field has the value PT_LOAD it means that this segment is 'loadable'. (other values this field can have is PT_DYNAMIC, which means that this segment contains dynamic linking information, PT_NOTE, which means the segment contains auxilliary notes, et cetera.) If the p_type field has the value PT_LOAD, the kernel must, in addition to knowing where the segment starts, also know how big it is, which is specified in the p_filesz field. There are also a couple of fields that describe where the segment is located in virtual memory space. However, the actual offset in virtual memory space is irrelevant for DSOs that are not linked, since they haven't been assigned a specific place in virtual memory space. For executables and so-called 'prelinked' DSOs (meaning that they have been bound to an executable even if they're dynamic), the offset is relevant.

However, even though the offset in memory is irrelevant for unlinked DSOs, the virtual memory size of the segment is relevant. This is because the actual memory space that the segment needs can be larger than the size of the segment in-file. When the kernel loads the segment into memory, if the requested memory size is larger than the segment size, the extra memory is initialized with zeroes. This is practical if there are so-called BSS sections in the segment. BSS is an old name for a section of data that contains only zero bits. Thus, as long as extraneous memory is initialized with zeroes, this is a good way to save space - you only need to know how large the bss section is, add that size to the current size of the segment, and the kernel handles the rest. An example of a BSS section is a section containing uninitialized variables in C code, since such variables are set to zero in C anyway.

Finally, each segment has a logical set of permissions that is defined in the p_flags field of the program header struct - whether the segment is writable, readable, executable or any combination of the three.

After this, the virtual address space for the ELF executable is set up. However, the executable binary at this point only contains the segments that had the PT_LOAD value in the p_type field. The dynamically linked segments are not yet loaded - they only have an address in virtual memory. Therefore, before execution can start, another program must be executed - the dynamic linker.

The dynamic linker is a program just like the executable we're trying to run, so it has to go through all the above steps. The difference is that the linker is a complete binary, and it should also be relocatable. Which linker is used is not specified by the kernel - it is contained in a special segment in the ELF file, which has the PT_INTERP value in the p_type field. This segment is just a null-terminated string which specifies which linker to use. And the load address of the linker should not conflict with any of the executables on which it is being run.

This ends the second part of the writeup. And there's plenty left..

Shared library writeup: Part 1

During my daily work this week, I found myself struggling with shared libraries, linking them, and the various compiler flags needed to make the type of library you want. I decided to actually learn this stuff once and for all, and so I am currently reading "How to write shared libraries" by Ulrich Drepper. I decided this was a perfect opportunity to multitask - both write stuff for the blog and learn something! Espec ially since you learn much better by writing about it. Hence, this will be the first part of my writeup of Drepper's paper.

In the most abstract, libraries are collections of code gathered into one file for easy reuse. They can be static, meaning that if you want to use the code in a program, the compiler must take the code contained in the library and bake it into the program upon compilation. Alternatively, they can also be shared or dynamic, meaning that they are not included in the program upon compilation, but the program contains mention of the libraries, so that on run-time, the program loads the library and incorporates it into the program.

Nowadays, (on Unix-like systems) libraries are handled by the so-called ELF (Executable Linkage format), which is a common file format that are used not just for libraries, but for executables and other types of files as well.

Earlier, other formats, such as a.out and the Common Object File Format (COFF) were used. The disadvantage with these were that when these libraries did not support relocation.

When you have a piece of compiled code (typically in what's called an object file), this file will contain a relocation table. Such a table is a list of pointers to various addresses within that object file, and these addresses are typically given relative to the beginning of the file (which is typically zero). When combining several such object files into one large executable, this object-file-specific list must typically be changed, since the object file now is not located at 'zero' anymore, but rather at some arbitrary point within the new executable.Then, when the executable is to be executed, the addresses are again modified to reflect the actual addresses in RAM. This last part is what is not supported by the old library formats.

This essentially means that each library must be given an absolute address in virtual memory upon creation, and that some central authority must keep track of where the various shared libraries are stored. In addition: when we make additions to a library that is supposed to be shared, we don't want to have to tell all the applications that used the old version that the library has changed - as long as the new version still contains all the stuff we need for our application, it should still work for that application without having to re-link the application with the new version of the library. This means that the table that points to where the various parts of the library are located must be kept separate from the actual library, and it must actually keep track of the pointer tables of all the old versions of that library - once a function had been added to a library, its address lasted forever. New additions to a library would just append to the existing table. In short, a.out and COFF were not very practical for use as shared libraries, although they did make the program run fast, since there is no relocation of table pointers at run time.

Enter ELF

ELF is, as mentioned before, a common file type for both applications, object files, libraries and more. It is therefore very easy to make a library once you know how to make an application - you just pass in an additional compiler flag. The only difference between them is that applications usually have a fixed load address, that is, the (virtual) memory address into which they are loaded upon execution. There is a special class of applications, called Position Independent Executables (PIEs) that don't even have a fixed load address, and for those, the difference between applications and shared libraries are even smaller.

For an application that contains no dynamic components (no shared libraries etc.), its execution is straightforward: The application is loaded into memory, then instruction at the 'entry point' memory address is executed, which should start a chain of events that ends with the termination of the program.

For applications that do contain dynamic components, it is less straightforward: There must be another program that can coordinate the application with the DSOs (Dynamic Shared Objects) before execution of the program starts.

The ELF file structure

ELF files usually contain the following:

• the file header
• the Program header table
• the Section header table
• Sections

Sections are the meat of the ELF file - they contain all the information about what is actually going on. The other structures are just there to organize the sections.

The section header table is a table with references to the various sections of the file. The program header table contains references to various groupings of the sections. So you might say that the section header table describes each 'atom' of the file, whereas the program header table collects these atoms into 'molecules' and makes sensible chunks, called segments, that are sections that work together to form a coherent whole.

End of part 1 of the writeup! And I'm only on page 3 of the paper!