Entries Tagged 'General' ↓

Miguel,
thanks for pimping
Swfdec. But I think your conclusions are wrong. Flash is
not even close to what I’d suggest as an option for “rich
media”.

For a start, the currently supported Video formats for Flash
are Sorensen Squeeze and On2 VP6. As far as I know, both of
those companies are actively fighting Free implementations
of the codec. That makes Flash video as bad or worse than
VC-1. You can play all 3 of them with FFmpeg btw.
Then there’s the fact that the interaction between Flash and
browsers is crap. You can execute Javascript from the Flash
file and you can execute predefined Flash functions from
Javascript, but there’s no DOM or anything. But what about
integrating Adblock, the “animate only once” option, popup
blocking or all the other things you want integrated?
As for the things you mentioned: The Flash spec is
semi-open, in the sense that you may not use it to write an
interpreter for Flash. That’s equivalent to closed for
everything you care about. And the spec covers the file
format only, not the video formats used or what script
objects are supported.
And you can populate Flash files using remote data. You can
even chose between XML and Flash. The XML needs to be
interpreted, but I’m pretty sure there’s convenience
libraries for you that do that.

And it’s not like the Free alternatives are far behind in
the game. HTML5
has a video tag and SVG is far superior to anything
Flash offers in drawing options. Especially because it’s
scriptable while Flash shapes are not.

There’s 2 things missing:

• Simple animation support: Flash makes creating
  animations extremely easy. Doing the same thing in
  anything Free (HTML, SVG, even Gtk) requires being a good
  code monkey. The example above required NO coding.

• Authoring tools for rich web applications. Depending on
  who you ask, the best Free authoring tool is either vim or
  emacs. There’s the Gimp and Inkscape of course, but neither
  does animations nor scripting, so it’s pretty useless for
  doing rich web apps.

So I can tell you what you want. You want to create an
awesome authoring tool that integrates HTML5 and SVG and
make Webkit and Mozilla handle those. And after that I bet
rich web apps are Free.

Talking about security is “funny”. Like looking how
people react to security issues or talking with other
developers in #gnome-hackers, where I recently claimed that
GNOME’s security sucks and got quite a backlash. Since
writing multimedia players like GStreamer or Swfdec made me
think about security (I’m still far from expert
unfortunately), I’d like to elaborate a bit on that.

Let me first mention one important thing: I don’t just
consider crashes to be stuff that shouldn’t happen.
Everything that disrupts the user’s computing experience
should not happen. Taking too many CPU ressources or too
much memory for too long effects other applications
negatively, so it should be avoided.

The biggest problem for me has always been the question of
trust. When interpreting data I have to ask myself if I
trust that data or not. If I have a number that says
“allocate this much memory”, I better trust it to report a
correct size or I’ll get an out-of-memory problem. A worse
problem are pointers: read (or write to) the memory pointed
to by this address. If you can’t trust a pointer you’ll
likely get a segfault. So you better trust every pointer in
your app. Note that this includes modifying pointers (like
array indexing), which should only be done with trusted
values. Stuff like that quickly gets my head spinning about
which values are trusted and which aren’t. Another example:
Do we trust this string to be valid UTF-8?

A lot of data is read from files these days. Now, what files
can application developers trust? Apparently we trust all
the data a user doesn’t have write access to, since I can
make apps crash easily by removing Glade files, icons or
linked-in libraries. But should applications trust stuff in
the user’s home directory?

Currently a lot of applications trust stuff in the user’s
directories unconditionally. Bash and X execute files in
there for example. However epiphany allows you to save files
from the web in there, too. What do you think clueless users
would do when a web site tells them to download a file named
.xinitrc? Probably nothing, since .xinitrc must be
executable in order to get executed, but we’re already very
close to running “rm -rf ~” on login here. And of course we
suddenly have a lot of untrusted files in the user’s home
directory. Nautilus for example knows this, since the
thumbnailing process has brought down Nautilus quite often
in the past. But I bet a lot of other applications don’t
think about that.

And then there’s the fact that it’s apparently dead
easy to find security holes in Free Software while Windows
products have been exploited by those issues in the
past.
So I guess when the Linux desktop gets interesting for
crackers, we’ll be in for quite a ride. And by “we”, I’m
talking about every application that reads files.

So I’ve managed to write software that a lot of people are
suddenly interested in. It’s an interesting thing to say the
least. However “becoming famous” was nothing like what I’d
had imagined. Here’s what happened after my initial
blogpost:

1. The media picks it up.
   The media is interested in new stories. So they post them.
   However, they often prefer to give those stories their own spin.
   (The really weird stories have luckily already disappeared
   from Google, so no links to them.) None of those journalists
   contacted me about it, they just wrote something.

2. The people pick it up.
   Lots of people start talking about it. How do I know? I
   googled. Only 5 people talked to me about it. Mostly they
   just went into obscure forums or chats to discuss it and
   complain about missing features. Almost nobody of them
   realizes that it’s a really good idea to come to me and ask
   questions. It’s highly motivating if someone shows interest
   in my product. Even if it’s just complaining that it doesn’t
   build on his box. And if you ask, there’s a high chance I
   get interested in solving your problem, especially if it’s a
   pretty simple problem.
   (Side note: The Ubuntu forums are
   lucky to have such a high Google ranking, so I could at
   least find their questions.)

3. When there’s a release, the distros pick it up.
   Distros are really fast at taking your source code and
   sticking it in the unstable branch of their package
   repositories. Lots of them, in particular the big ones,
   contact you to tell you about it and ask questions. The
   process of integrating new software into distros seems to
   work really well. Congrats from me to the distro people for
   that. Great work you’re doing there. In particular Gentoo.

4. People start blaming
   They seem to read everything I have said in the past and
   spin it in their direction. I’m suddenly personally
   responsible for not working on Gnash and fragmenting the
   Free Flash movement, I sure as hell have to provide
   OpenBSD support or my software is crap, stuff like that.
   Again, noone comes and asks, that would be far too
   complicated. And probably wouldn’t be compatible to their
   predetermined opinion.

5. Hackers don’t show up
   I was expecting some interested people to show up and have a
   look at the code. In particular because the code is
   documented far better than all my previous code (including
   GStreamer ;)). I think there have been 2 people so far that
   looked at it. But that’s about it. Probably everyone is busy
   doing their own project. Or people still think doing Flash
   is hard (hint: it’s not).

But then, the few people that have shown up and talked to me
have made a lot of things happen and I guess I’ll cut a new
release soon to make those people happy. In particular:

• People wanted an easy API to embed Flash files. So I
  created libswfdec-gtk. You should be able to play Youtube
  videos in your Gtk app with 20 lines of C.

• Or you could do it in 10 lines of Python with the new Pyswfdec Python
  bindings that Gian Mario
  Tagliaretti has been working on.

• Last but not least I’ve managed to use GStreamer instead
  of ffmpeg/mad for video and audio decoding. While that
  doesn’t sound to exciting at first, it means you can build
  swfdec now without linking to patented multimedia formats
  and still be able to enjoy Youtube. I know at least Fedora
  was very interested in that.

No new work on Flash features so far, since I’ve been
working on a branch redoing the ActionScript interpreter.
Once that hits (that’ll definitely be after next release),
it’ll make a huge number of Flash files supported. But
that’s still some work.

What I’ve been wondering and like to get some feedback from
people active in this area is if it’s a good idea to export
the script interpreter. ActionScript is
almost the same as Javascript (same syntax), so it offers a
sandboxed execution environment. In this case completely
GObject’ified. So you could use it if you wanted to script
Flash files, or you could even add hooks to it in your own
app and download interesting scripts from the web. Think Get Hot new Stuff
with scripts. To me that’s somewhere between total crack and
really exciting.

It’s done. Swfdec
0.4.3 is out, so for everyone that couldn’t or didn’t
want to dig into the dirty world of unreleased code, grab
the releases of Swfdec
and Swfdec-Mozilla.
It will probably take a while before this hits the big
distributions, since there’s legal issues with the video
plugins and Debian and Ubuntu are still busy releasing.

For this release I’ve done all the easy tasks in Youtube
playback, so you can play/pause, rewind and watch the slider
on the seek bar. You still can’t seek or change the volume,
but that has to wait. But you can right-click, select
Properties and save the video you’re watching. So there, a
feature that’s not in the official player.

A big thanks to all the people who’ve encouraged me, filed
bugs or submitted patches, wrote articles or blog posts
about swfdec and otherwise kept me busy for the last week.
It’s been an interesting experience although I still think I
suck at marketing and public relations. Somehow Christian
does a much better job pimping my stuff.

The only thing I’ve figured out is that even though lots of
people are interested in Flash in many different forms,
there’s a very small amount of people actually hacking on
making Free Flash a reality. Even though the technologies
used in Flash are what everyone wants. I wonder why that is?

So here’s a small update to last week’s Swfdec
Youtube post. I’ve spent most of the time on the awesome
IRC feedback that I got. Thanks everyone. Here’s some
answers to the most common questions I got:

Swfdec plays Youtube. Now what does that
mean?
It means that when you grab the Swfdec library and the
swfdec-mozilla package out of git and manage to install it
correctly, you will be able to go to any Youtube video site
and have it play back the videos in your browser. The
buttons don’t work yet and it certainly doesn’t behave 100%
like the Adobe plugin, but it certainly plays the videos.

OMG, swfdec doesn’t work on 64bit machines!
OMG, it does! Since I posted the last entry, I’ve got lots
of offers for remote access to 64bit machines and spent the
last hours making sure it compiles and passes the testsuite.
Of course I have no 64bit machine and until 3 hours ago
didn’t know most of the things I’d need to look for, so
expect some bugs being left. Please test if you’re on 64bit
and if you find crashers, file bugs.

Where’s the release?
I intend to do a release soon, but I’d like to have a
working pause button and slider in Youtube. I’m hoping for a
release at the end of this week. For now, use git.

How close is Swfdec to being a complete Adobe Flash
replacement
That really depends on your definition of close. For the
definition “implements all of Flash’s features” it’ll
probably not hit 5%. For the definition of “plays all the
Flash files on the Web” I think it’s 80/20 right now. Swfdec
plays 80% of the ads and 20% of the real content. And no,
that wasn’t on purpose. I’ve been following a simple rule
for what to implement: I take an interesting flash file and
make it work correctly. So if there’s any Flash files that I
might find interesting and that you want to have working in
Swfdec, don’t hesitate to contact me with URLs to those
files. I’m always looking for cool Flash files.

What can be expected from Swfdec in the
future?
I have no idea. I’m looking for a job at the moment and
since that job will most likely not be me hacking on Swfdec
fulltime, expect the Swfdec development to slow down in the
future. However, it only took me half a year to get Youtube
working and Youtube is already quite complicated. So in
short: I have no idea.

YOOOUUUUUTUUUUUBE

GO Swfdec!

6 months of work, I’m now gonna go partying.
See you at Cebit.
PS: For the brave people: The code is in git.

Miguel
links to a
blog post by Anne Zelenka (read the comments!) about how
open Flash is and how good an open Flash would be. I think
one point they all don’t see in their “an open spec would be
enough” is the work necessary to produce an open spec.

Now let me get this straight. Adobe has a spec for the SWF
format, and they have lots of documentation on Actionscript,
so producing some spec would be easy. But the problem is
that the existing specs only describe correct behavior, but
not the more important part on how to treat errors. Consider
an example where the spec might say something like “height:
Integer – the height of the current movie”. So what happens
when your code does height = new Object()? It’s written
down nowhere. In the current closed world, the solution is
easy: the accepted behavior is what the Adobe Flash player
does. So if you want to write an open Flash Player like Swfdec, you don’t
need a spec, you need patience and lots of test cases.
Because there’ll surely be a Flash somewhere that does
height = new Object () or height
= height / 0 or height = "Hello World" or…

A good example of how hard it is to handle the unexpected
right is Acid2
for CSS. And in the case of Acid2, there even exists a spec
about how to handle all the errors. (I was going to link to
something I read by I think Håkon Lie about why having a
defined way of handling errors is important as opposed to
just aborting, but I can’t find it in Google.) And error
handling mechanisms are an important part of an
implementation. The
Mozilla team needed 1.5 years to correct their error
handling. So if you figure out something new and exciting
about Flash, it can easily mean you have to redesign a large
part of your player. So it’s important to know beforehand
and should be part of the spec.

And while we’re talking about necessary rewrites: A part
that no spec talks about is implementation complexity. If a
function is O(1) in the official player while it is O(N) in
yours, you have a problem when someone calls it excessively
in a loop. And then there’s probably code relying on
timeouts, data input or the phase of the moon.

Another thing I’ve been wondering about lately is the
complexity of implementing a standard. I have no clue how
Flash relates to SVG in complexity, but SVG has an open spec
and that one is 4 years old. Do we have any SVG compliant
implementations by now? HTML has a free spec, too. It took
the Mozilla team 6 years from open
sourcing to a
release for their browser. So if a complete Free
Flash specification started to exist tomorrow, would it take
5 years to implement?

An open Flash spec would definitely make Flash inch closer
to World Domination, but it’d still take a very long time to
make it really Free. Open sourcing the player would probably
make that happen way faster.

I did it

So I’ve just managed to finish my Christmas present
only a week late, but anyway, after 3 months of not hacking
on anything else, I’ve finally released Swfdec
0.4 and its companion, Swfdec
Mozilla 0.4.

The good news: The mozilla plugin is now almost 100%
compatible with South Park Studio. So you can do
this.
(Yes, that’s an SVG taken from the Flash)
Unfortunately there’s no UI for it yet. Implementing Flash
is kind of a huge task afterall.

And finally, here’s the disclaimer: This software is not
rock-solid. So if the plugin kills your browser, don’t blame
me. File a bug or
talk to
the list instead.

Happy new year!

So, talking about the MS/Novell deal has been declared
stupid by dobey
even though according to other hackers 80 percent of hackers
think that there is nothing wrong with it. jclinton
has issued a statement condemning this action. Perhaps he
said it best

“Since you guys are all buddy, buddy
and everything, I’m sure it wouldn’t be too hard for you to
take a few minutes of his time to explain the spirit of Free
Software to Steve.”

Personally I think that if
people weren’t outraged about it, we’d have already lost to
MS.

Update: Iain
has
said that if more people think like this he will do what he
is told and stop blogging completely.

Flash

So about a month ago Federico blogged a Youtube link again.
That made me wonder about Free Flash decoders and I had a
look around. In the end I started hacking on David‘s Swfdec, since it
has all the goodies: Cairo for beautiful
Graphics, GStreamer for
sound, David’s excellent code (the SWF parsing code is
awesome) and of course it was written in Gnome C.

Long story short: I haven’t had this much fun in hacking for
ages. I think I haven’t done much else in the past month but
hacked on Swfdec and watched lots of stupid Flash movies
lots of times.

So what’s the current state? It’s still very far from
finished since Flash is huge. It’s a graphics framework
(think GDK) with a scripting engine (think Javascript). But
if you’re adventurous, bored or both, feel free to check out
the sources from git://people.freedesktop.org/~company/git/swfdec
and play with them. They might even play back some
weird flash file. For now I’ll continue to make FLash
files work one by one. Next stop: South Park.

PS: Before someone asks: Gnash is low quality, written in
C++ with Boost, requires Copyright assignment, svn doesn’t
even compile and it does less than my current Swfdec, so I
didn’t hack on it.