Tag: pet food

Well nobody seems to comment on ‘good stories’ – maybe I should rant more often? Anyway, it seems I have a reputation in the GNOME world as being an arsehole, so why not. Threat’s about employers reading the blog? Yeah nice one.

I realise I was going to offend the author of packagekit – but seriously, this is not ready for release as the main package ui on any distro. If what it did it did well it might be ok, but it has some time to go. Maybe he needs some offending to get his arse into gear and make it happen and prove me wrong? It’s out there in the wild now as the primary update interface on a public release of a popular distribution – he’s gotta expect criticism, and he’s gotta expect at least some rants – it’s not like i’m mailing the guy or spamming his blog’s comments – it’s my blog. If the app is busy, it needs to make that obvious, not just sit around for 10’s of seconds appearing to do what you asked, and then come up with a blank list for no apparent reason. I have the fastest net connection I can buy but it was a bit busy at the time – still, why does packagekit not cache any meta data, at least for the current session? How come it takes 100mb to run yum – if that’s all it’s doing – I thought surely it was doing more than that? Installing 1 package at a time is not good enough – computers are designed to run batch processes automatically, why force me to handle it? Why is every operation serialised when many don’t need to be? The machine was fine when running yum by itself (even with the ‘busy’ network), so it isn’t the cpu/memory or network (it wasn’t swapping even running the update thing). You can complain that I need a faster box or net – but I don’t need either for running anything else I want to run.

I’ll just say one more thing – you can’t have it both ways – if it wasn’t ready for prime time you should have asked Fedora not to use it – or by getting it in the distribution you get the exposure and fame and flashing lights – but have to expect the exposure to generate a range of opinions, from positive to negative. It isn’t personal (how could it be, we don’t know each other) – although it is impossible not to take it that way I know.

Ubuntu is mostly fine – but after giving it a pretty good run I think it’s just not for me. It’s too focused on newbies or windowsies, not ‘veteran’ linux users – yes, that is their target of course, but attracting developers wouldn’t hurt them either. Venting my personal frustration in my own blog should be ‘allowed’, and I don’t think I need to ask anyone for permission. Debian is known for making strange decisions by an overly politicised process by strong-willed individuals – my opinions there are nothing new (and that is all I meant by ‘*BSD like’ btw).

And yes Synaptic is quite nice. The only thing I don’t really like about it is that it’s quite slow at searching, and listing packages. I can’t remember what I used on suse (10), but from (a somewhat unreliable) memory i thought it was faster/nicer to use.

Umm, if the network cable comes loose, I’d presume the network would come back online all by itself when it got plugged back in – just like it always has? I certainly shouldn’t be logged in and running a crapplet (NB: i didn’t invent the word) for it to reconnect for example, or run any command as root. Do you run a desktop on a web server just to configure it’s network? Can’t their cables also get knocked out?

I installed ‘everything’ (desktop, web server, developer) but of the 3 applications I use daily on any computer – one wasn’t installed. Of course i’m going to complain about some fluff that is installed that directly affects my user experience that was installed instead – hey at least the man pages and info files are there. And are you saying packagekit is not installed by default in most configurations? I’d never heard of it – how could I go looking through dozens of packages to remove it? I imagine any ‘desktop’ would include the other stuff too. I’ve been there and done that – trying to tune my system exactly how I wanted before I installed it. All I did was end up with a broken system and wasting even more time fixing it at both ends. I rarely even run the disk partitioner anymore either,since i’ve had more than one failed install by trying to get things how I wanted it.

As for mono – I don’t hate mono. Mono is ok, technically quite a feat too – I think the effort could have gone elsewhere personally (hint: that means it’s opinion), but although it might not be obvious, I do know where Miguel is coming from and I ‘get’ what he is trying to do, and it is a good thing that someone is doing it, and that they have plenty of financial backing to pursue it, and the grand vision and enthusiasm for it. Ok, initially I was lumbering with evolution, bonobo and e-tree and thought he was just starting another ill-fated quick-results project he’d leave for someone else to finish! But that was the very early days – I worked on a mono plugin for Evolution after all – but nobody seemed to want it so I gave up. However he had to expect political backlash from many people given he was effectively cloning MS technology … and much later on the novell-ms deal didn’t help – no matter what the (secret) realities are of it, it’s done, and it will always be hanging over the project for some people – WHICH IS A TERRIBLE SHAME because so much time and effort has gone into it and there are plenty of good ideas and technology there. Politics in and around technical projects totally sucks, but that’s the reality. Time will tell anyway – and tends to iron out issues like this by itself.

For me, currently there’s no apps I need that use it (f-spot is a really good app but i dont need it), and I want to avoid the temptation to write .net code at home (odd yes, but indeed true). And yes I do have personal misgivings about any MS technology in general, and specifically any on my Linux box, but that was just the icing on the cake. What I hate is .NET itself. I use it at work. On a Windows box. That’s a lot to hate. At least with mono there’s the potential that one day they’ll be able to address the main memory issue with any vm based system – of having multiple virtual machines running for separate applications. Would putting them all in 1 environment work? They do it for Java for enterprise/backend applications – how come it isn’t used for desktop applications as well? If the whole desktop and most of the applications ran from a single vm it would probably do rather well – probably better than ‘n’ C applications which have to initialise each application toolkit separately and which share only read-only c library data and code between them (and an order of magnitude better than ‘n’ C language engines loading (and compiling) ‘n’ language ‘scripts’ which load ‘m’ language ‘libraries’ and sharing only the memory-mapped C library parts between them). (where by ‘C’ I means pre-compiled memory-mappable code/data).

I do rather dislike python mind you. Somewhat how I dislike visual basic. I don’t really hold strong opinions about any other languages but those two. Well, javascript isn’t that high on my list either, tcl has some issues at that. On reflection maybe the reason is simple – the generally bad experience (IN MY EXPERIENCE – it’s called opinion, not everyone agrees with that opinion, but it’s mine, and I hold it) from vb/python apps. And well, BASIC sucks.

BTW with Fedora, my install was still left with init running at level 3 (vi fixed that). I’m not sure I ever told it I didn’t want X running (I installed a desktop system afterall). Maybe it had something to do with a text-mode install, but I can’t remember being given the option to turn off/on X (apart from setting it up).

Made good progress over the weekend on the mplayer vo module. Took a
bloody long time though – bugs in code, not undertanding api’s, not
understanding how mplayer wants it’s vo drivers to work. But I got
there in the end – well just, it had me solidly occupied the whole
weekend, much to the chargrin of my housemates who wanted to play
games. Even managed to watch a couple of movies to test it out –
although I still can’t work out how to stop the fucking screen
blanking every 10 minutes under ubuntu (or stop the log output to the
virtual console) (yeah i googled, but couldn’t find any way to do it
programmatically – i can unblank but not disable it).

It can upscale an SD source to 1080p without problems (and without
tearing, oh happy day) – it is only bi-linear scaling, but that’s good
enough for now. There’s still plenty of scope for improvement –
e.g. now I need to work on a job queue mechanism rather than
loading/executing the whole spu programme each frame.

Although I knew dma’s needed to be 16 byte aligned I kept forgetting
to start with – the cause of much frustration and cursing. I also had
problems with the spu seeming to crash after a short time. After
about 200 frames it would just die but the rest of mplayer kept
running and executing the spu programme returned no error. I couldn’t
work out what was going on. It seems that once you load a programme
into the spu context, it isn’t really kept around forever – so it
would semi-randomly dissapear at some future point in time. I think
this is something I’d previously ‘discovered’ but forgotten about
since I last did any spu coding.

It was also a little difficult finding decent documentation on how to
embed spu binaries into ppe code, it often assumed you knew how,
didn’t go into that detail, or used some magic makefiles not specified
directly. I found make.footer in the sdk examples at last, and after
some manual runs distilled that into my own makefiles.

Although I had bugs in the spu code I had written/compiled before I
tried running it, at least they were not terminal issues. The
algorithms were sound, I was just out in little ways or just forgot to
finish bits of them. The YUV converter seems to need to do an awful
lot of work, but I guess that’s why it’s so slow without an spu. The
actual calculation is easy and simple, but clamping 64 values to
[0,255] just takes a lot of instructions for example. So far i’m
sticking with a packed ARGB internal format in memory and 4xeach
colour component/planar when in registers – the loads vs the
loads/shuffle’s aren’t much different, but I’ll try an unpacked 32 bit
int/float planar/interleaved as well at some point. Took me some time
to get the nearest pixel horizontal scalar working – but that’s
because I tried to do too much in my head/by hit and miss without
writing it down and working out the bit numbers properly – i got the
main addressing working but stuffed up the sub-word addressing.
What’s nice is that all the calculations (and memory loads) are the
same for a linear scaler, it then just needs to add a pixel blending
step afterwards – so that fell out very easily once I got the
nearest-pixel scaler working.

The basic algorithm is a very simple one which used fixed-point
integer arithmetic so the inner loop is merely and add, with a shift
used to perform addressing.

void scalex_nearest(unsigned char *srcp, unsigned char *dstp, int inw, int outw)
{
  int scalexf = inw * (1<<CCALEBITS) / outw;
  int sxf = 0, dx = 0;

  while (dx < outw) {
    int sx = sxf >> SCALEBITS;

    dstp[dx] = srcp[sx];

    sxf += scalexf;
    dx += 1;
  }
}

It may not be as accurate as it could be, but so far it’s accurate enough for what I need.
And it’s branchless which is important for vectorising.

Converting this to vector/simd code is straightforward if not entirely trivial. And the spu has
some nice instructions to help – as you’d expect from a simd processor.

Basically the code calculates 4 adjacent pixels at once – the above
calculation is basically repeated 4 times, with a single pixel offset
for each column.

The basic scale value is the same, but it is a vector now:

  vector int scalexf = spu_splats(inw * (1<<CCALEBITS) / outw);
       // spu_splats - copy scalar to all vector elements

But the accumulator needs to be initialised with 4 different values –
each as if it were 1 further in the loop. i.e.

  vector int sxf = (vector int) { 0, scalexf, scalexf*2, scalexf * 3 };

And each loop we need to increment by 4 (this is a silly little detail
I knew about but forgot to put in the code assuming it was there –
much frustration).

  scalexf = spu_sl(scalexf, 2);   // spu_sl - element wise shift-left

It is in the inner loop where things get a bit more complicated. We
can only load single quadwords at a time from a single offset, so the
algorithm needs a bit of tweaking. Instead of loading 4 separate
addresses which the original algorithm would require, we can take
advantage of the fact we are only scaling up (i.e. we will never need
to access more words than we’re filling) and use the shuffle
instruction to perform sub-word addressing. The basic addressing thus
remains the same, and we just use the first value for this. This
gives us a base address from which we load 2 quadwords – the
subaddressing will always fit within these 8 values (we need 8 values
since we may have overlapping accesses). We are also referencing 4
values at once, so we have to multiply the address by 4 – or shift by
2 less (12 vs 14)

Explanation of this by way of example:

  example: scale from 10 to 15 in size, use 14 bits of scale
  input  :  [ 1, 2, 3, 4, 5, 6, 7, 8, 9, a] (in words)

  scalexf will be 10 * (16384) / 15 = 10922

  loop0:
      sxf = [      0, 10922, 21844, 32766 ]

  address = sxf[0] >> 12 = 0
    load0 = [ 0, 1, 2, 3 ]
    load1 = [ 4, 5, 6, 7 ]

  loop1:
      sxf = [  43688, 54610, 65532, 76454 ]

  address = sxf[0] >> 12 = 10 = 0 (quadword aligned)
    load0 = [ 0, 1, 2, 3 ]
    load1 = [ 4, 5, 6, 7 ]

sxf needs to be remapped to a shuffle instruction to load the right pixels. If each element of sxf is shifted right by 14 we have:

   offsets = sxf >> 14 = [ 0, 0, 1, 1 ]

This needs to be mapped into a shuffle instruction to access those words. The shuffle bytes instruction takes 2 registers and a control word. The registers are concatenated together to form a 32 byte array, and the control word looks up each byte at a time into this array. i.e. we need to get a shuffle pattern:

  pattern = [ 0, 1, 2, 3, 0, 1, 2, 3, 4, 5, 6, 7, 4, 5, 6, 7 ]

If result is the result vector represented as an array of 4-byte words, and source is the two source vectors represented as an 8 element array, then using this pattern is equivalent to:

   result[0] = source[0]
   result[1] = source[0]
   result[2] = source[1]
   result[3] = source[1]

The easiest way to do this is to duplicate each offset stored in the lsb of the words in the sxf register into every other byte in the same word. Shift everything by 2 (*4) (or do it initially), and then add a previously initialised variable which holds the byte offsets for each word. i.e.

   tmp =  offsets << 2   = int: [ 0, 0, 4, 4 ]
   pattern = shuffle (tmp, tmp, ((vector unsigned char ) { 3, 3, 3, 3, 7, 7, 7, 7, 11, 11, 11, 11, 15, 15, 15, 15 }) = char: [ 0, 0, 0, 0, 0, 0, 0, 0, 4, 4, 4, 4, 4, 4, 4, 4 ]
   pattern = pattern + spu_splats(0x00010203)  = char: [ 0, 1, 2, 3, 0, 1, 2, 3, 4, 5, 6, 7, 4, 5, 6, 7];

Which is the desired shuffle pattern. Getting the result is then a simple shuffle:

  dstp[dx] = spu_shuffle(load0, load1, pattern);

That is the basic algorithm anyway – there are some added complications in that the offsets aren’t going to be just sxf shifted except for the first column, and they need to be relative to the quadword addressing. So the offset calculation is a little more like:

    addr = sxf [0] >> (14-4+2) // this is the byte offset address of the quad-word we're to load (pass this directly to lqx instruction, it will ignore the lower 4 bits for free)
  offset = addr & (-16)          // mask out the lower 4 bits
  offset = addr - offset[0]          // find out the offsets of each member of the vector relative to the first one's actual address (shuffle is used to copy offset[0] to all members)
  offset = offset >> 2         // only the upper 2 bits since we're addressing 4-member vectors

So for both loops of the example:

loop0:
    addr = sxf >> 12    = [  0,  2,  5,  7 ]
  offset = addr & (-16)   = [  0,  0,  0,  0 ]
  offset = addr - offset[0]   = [  0,  2,  5,  7 ]
  offset = offset >> 2;   = [  0,  0,  1,  1 ]

     tmp = offset << 2  = [  0,  0,  4,  4 ]
 pattern = shuffle(tmp)       = [ 0,0,0,0, 0,0,0,0, 4,4,4,4, 4,4,4,4 ]
pattern += splats(0x01020304) = [ 0,1,2,3, 0,1,2,3, 4,5,6,7, 4,5,6,7 ]

loop1:
    addr = sxf >> 12    = [ 10, 13, 15, 18 ]
  offset = addr & (-16)   = [  0,  0,  0, 16 ]
  offset = addr - offset[0]   = [ 10, 13, 15, 18 ]
  offset = offset >> 2;   = [  2,  3,  3,  4 ]

     tmp = offset << 2  = [  8, 12, 12, 16 ]
 pattern = shuffle(tmp)       = [  8 (*4), 12 (*4), 12 (*4), 16 (*4) ]
pattern += splats(0x01020304) = [ 8,9,a,b, c,d,e,f, c,d,e,f, 10,11,12,13 ]

And writing this down I now see there’s a couple of redundant shifts happening,
so either my memory of the code is wrong, or I can simplify the code here

And the result of 2 loops would be:

  output = [ 1, 1, 2, 2, 3, 4, 4, 5 ] (in words = 4xbytes each)

So put it all together and that’s about it (ok, strictly you need to worry about
masking out the last results beyond what you want, but i’ll assume the input
and output size are both multiples of 4). Now the nice thing about
using the fixed point calculation is that it already gives you the
fractional part of the address as a by-product, and in a form suitable
for our calculations. So adding linear interpolation is at least in
this sense, free. All we need to do is load up a vector which has
every *next* pixel in it, and then interpolate based on the fractional
part in vsxf. We can use the shuffle pattern previously obtained,
offset by 1, to get the second value – it will always address within
the 8 values already loaded so we needn’t even perform another memory
load.

  patternnext = pattern + spu_splats(0x04040404)     // address the next word of every word
  valuenext = spu_shuffle(load0, load1, patternnext) // and we have it

  // perform blending - valueout = value0 + ( value1 - value0 ) * scale
  diff = valuenext - value                           // get signed difference
  scale = vsxf & 16383                      // mask out fractional part
  offset = (diff * scale) >> 14           // perform fixed-point fractional multiply
  valueout = value + offset                       // and that's it

(in reality since i’m storing packed format, I need to first unpack
the argb values into separate planes of each component, perform the
diff/scale/offset/valueout calculation once for each component, and
then re-pack the data).

Things to think about: If the code used shorts it could process 8
values at once instead of 4 – except the multiplies can only process 4
at once, so it would complicate these operations a little and require
additional packing instructions. It would probably be a win if the data was
stored as 16 bit fixed-point integer planes. Floats could be used – which
simplifies some of the work (but not much), but complicates the
addressing slightly, since it needs to be converted to integers for
that. Floats would only make sense if it were used as an intermediate
buffer format, and we were working on unpacked arrays of bitplanes.
Also note that the spu has no integer division, so the initial setup
of scalexf is messy – I did this using floats and then converted to
integers to do the work – either way it isn’t a big issue since it
only needs to be done once (using floats creates smaller code). Many
of the values could be pre-calculated with a single row of data,
although this then turns some instructions into loads, which may flood
the load/store pipeline. Maybe some combination could be used,
e.g. calculate the blending values/addressing manually and load the
shuffle values from a table.

Dunno what this fucked editor has done to the alignment – everything aligned when i typed it into emacs. And for some stupid reason, html input isn’t html – it honours carriage returns too. I give up.