academia – muellis blog

Talking at ARES 2019 in Canterbury, UK

It’s conference season and I attended the International Conference on Availability, Reliability, and Security (ARES) in Canterbury, UK. (note that in the future, the link might change to something more sustainable)

A representative of the Kent University opened the event. It is the UK’s European University, he said, with 20000 students, many of them being from other countries. He attributed that to the proximity to mainland Europe. Indeed it’s only an hour away (if you don’t have to go back to London to catch a direct Eurostar rather than one that stops in, say, Ashford). The conference was fairly international, too, with 230 participants from 33 countries. As an academic conference, they care about the “acceptance rate” which, in this case, was at 20.75%. Of course, he could have mentioned any number, because it’s impossible to verify.

The opening keynote was given by Alistair MacWilson from Bletchley Park. Yeah, the same Bletchley Park which Alan Turing worked at. He talked about the importance of academia in closing the cybersecurity talent gap. He said that the deficit of people knowing anything about cybersecurity skills is 3.3M with 380k alone in Europe, but APAC being desperately short of 2.1M professionals. All that is good news for us youngsters in the business, but not so good, he said, if you rely on the security of your IT infrastructure… It’s not getting any better, he said, considering that the number of connected devices and the complexity of our infrastructure is rising. You might think, he said, that highly technical skills are required to perform cybersecurity tasks. But he mentioned that 88% of the security problems that the global 5000 companies have stem from human factors. Inadequate and unfocussed training paired with insufficient resources contribute to that problem, he said. So if you don’t get continuous training then you will fall behind with your skill-set.

There were many remarkable talks and the papers can be found online; albeit behind a paywall. But I expect SciHub to have copies and authors to be willing to share their work if you ask. Anyway, one talk I remember was about delivering Value Added Services to electric vehicle charging. They said that it is currently not very attractive for commercial operators to provide charging stations, because the margin is low. Hence, additional monetisation in form of Value Added Services (VAS) could be added. They were thinking of updating the software of the vehicle while it is charging. I am not convinced that updating the car’s firmware makes a good VAS but I’m not an economist and what do I know about the world of electric vehicles. Anyway, their proposal to add VAS to the communication protocol might be justified, but their scenario of delivering software updates over that channel seems like a lost opportunity to me. Software updates are currently the most successful approach to protecting users, so it seems warranted to have an update protocol rather than a VAS protocol for electric vehicles.

My own talk was about using the context and provenance of USB-borne events (illegal public copy) to mitigate attacks via that channel. So general idea, known to readers of my blog, is to take the state of the session into account when dealing with events stemming from USB devices. More precisely, when your session is locked, don’t automatically load drivers for a new USB device. Your session is locked, after all. You’re not using your machine and cannot insert a new device. Hence, the likelihood of someone else maliciously inserting a device is higher than when your session is unlocked. Of course, that’s only a heuristic and some will argue that they frequently plug devices into their machine when it’s locked. Fair enough. I argue that we need to be sensitive and change as little as possible to the user’s way of working with the machine to get high acceptance rates. Hence, we need to be careful when devices like keyboards are inserted. Another scenario is the new network card that has been attached via USB. It should be more suspicious to accept that nameserver that came from the new network card’s DHCP server when the system has a perfectly working network configuration (and the DHCP response did not contain a default gateway). Turns out, that those attacks are mounted right now in real-life and we have yet to find defences that we can deploy on a large scale.

It’s been a nice event, even though the sandwiches for lunch got boring after a few days 😉 I am happy to have met researchers from other areas and I hope to stay in touch.

AMCIS Towards inter-organizational Enterprise Architecture Management – Applicability of TOGAF 9.1 for Network Organizations

First of all, there is a LaTeX template for the ACMIS conference now. I couldn’t believe that those academics use Word to typeset their papers. I am way too lazy to use Word so I decided to implement their (incomplete and somewhat incoherent) style guide as a LaTeX class. I guess it was an investment but it paid off the moment we needed to compile our list of references. Because, well, we didn’t have to do it… Our colleagues used Word and they spent at least a day to double check whether references are coherent. Not fun. On the technical side: Writing LaTeX classes is surprisingly annoying. The infrastructure is very limited. Everything feels like a big hack. Managing control flow, implementing data structures, de-duplicating code… How did people manage to write all these awesome LaTeX packages without having even the very basic infrastructure?!

As I promised in a recent post, I am coming back to literature databases. We wrote a literature review and thus needed to query databases. While doing the research I took note of some features and oddities and to save some souls from having to find out all that manually, I want to provide my list of these databases. One of my requirements was to export to a sane format. Something text based, well defined, easy to parse. The export shall include as much meta-data as possible, like keywords, citations, and other simple bibliographic data. Another requirement was the ability to deep link to a search. Something simple, you would guess. But many fall short. Not only do I want the convenience of not having to enter rather complex search queries manually (again), I also want to collaborate. And sending a link to results is much easier than exchanging instructions as to where to click.

Proquest
- Export to RIS with keywords
- Deeplink is hidden, after “My Searches” and “actions”
Palgrave
- Export as CSV: Title, Subtitle, Authors/Editors, Publication, Date, Online, Date, Ebook, Collection, Journal, Title, ISBN13, ISSN, Content Type, URL
- No ability to link to a search
Wiley
- Export possible (BibTex, others), with keywords, but limited to 20 at a time
- Link to Search not possible
JSTOR
- Deeplinks to a search are possible (just copy the URL)
- Export works (BibTeX, RIS), but not with keywords
EBSCO
- Link to search a bit hidden via “Share”
- No mass export of search results. Individual records can be exported.
bepress
- Linking to a search is possible
- Export not possible directly, but via other bepress services, such as AISNet. But then it’s hidden behind “show search”, then “advanced search” and then you can select “Bibliography Export” (Endote)
Science Direct
- Not possible to link to a search. But one can create an RSS feed.
- But it export with Keywords
Some custom web interface
- Export with Keywords: ?
- link to a search

On the paper (pdf link) itself: It’s called “Towards inter-organizational Enterprise Architecture Management – Applicability of TOGAF 9.1 for Network Organizations” and we investigated what problems the research community identified for modern enterprises and how well an EAM framework catered for those needs.

The abstract is as follows:

Network organizations and inter-organizational systems (IOS) have recently been the subjects of extensive research and practice.
Various papers discuss technical issues as well as several complex business considerations and cultural issues. However, one interesting aspect of this context has only received adequate coverage so far, namely the ability of existing Enterprise Architecture Management (EAM) frameworks to address the diverse challenges of inter-organizational collaboration. The relevance of this question is grounded in the increasing significance of IOS and the insight that many organizations model their architecture using such frameworks. This paper addresses the question by firstly conducting a conceptual literature review in order to identify a set of challenges. An EAM framework was then chosen and its ability to address the challenges was evaluated. The chosen framework is The Open Group Architecture Framework (TOGAF) 9.1 and the analysis conducted with regard to the support of network organizations highlights which issues it deals with. TOGAF serves as a good basis to solve the challenges of “Process and Data Integration” and “Infrastructure and Application Integration”. Other areas such as the “Organization of the Network Organization” need further support. Both the identification of challenges and the analysis of TOGAF assist academics and practitioners alike to identify further
research topics as well as to find documentation related to inter-organizational problems in EAM.

FTR: The permissions I needed to give away were surprisingly relaxed:

By checking the box below, I grant AMCIS 2013 Manuscript Submission on behalf of AMCIS 2013 the non-exclusive right to distribute my submission (“the Work”) over the Internet and make it part of the AIS Electronic Library (AISeL).
I warrant as follows:

that I have the full power and authority to make this agreement;

that the Work does not infringe any copyright, nor violate any proprietary rights, nor contain any libelous matter, nor invade the privacy of any person or third party;

that the Work has not been published elsewhere with the same content or in the same format; and

that no right in the Work has in any way been sold, mortgaged, or otherwise disposed of, and that the Work is free from all liens and claims.

I understand that once a peer-reviewed Work is deposited in the repository, it may not be removed.

On Academia…

A paper that I have authored has ~~recently~~ been published a while ago, but I’ve put this post off for a long time now. Before talking about the paper itself, I want to talk about Academia as I have the feeling that I need to defend myself for playing their game™. The following may sounds overly pessimistic and a while a few bright spots are going to be mentioned, many have been left out for ranting reasons. Keep that in mind when reading that somewhat unstructured rant…

Published papers are the currency in Academia. The more you have, the more respected you are. The quantity is the main metric. No wonder, given that quality control measures are not very well deployed. Pretty much the only mechanism to ensure quality is peer review. The holy grail.

Although the more papers at “better” conferences or journals you have, the better you are, the quality of the conference or journal and the quality of the paper are rarely questioned after the publication. Again, I don’t have proper proof for the statements I make as this is supposed to be a more general rant on current practises in Academia. I can only tell from experience. From me listening to people talking about fellow academics, from observing key metrics in various web portals, or seeing people applying for academic positions. Those people usually have an enumeration of their publications. Maybe it’s a “selection”. But I’ve never seen that people put a “ranking” of the quality of the publisher nor the publication itself. And it wouldn’t make sense, because we don’t have metrics for that, anyway. Sure, there are some people or companies trying to come up with something meaningful. But metrics such as “rejection rate”, “number of citations”, or “h-index” are inherently flawed. For many reasons. Mainly because the data is proprietary. You have to rely on the conference or the journal providing you with correct data. You cannot know whether it is correct as there is no right for you to know. Secondarily, the metric might suffer from chilling effects, such that people think the quality of their publication in spe is too weak to be able to be published on a “high ranked” conference. So they don’t even bother to submit. Other metrics like the average citation count after five years resembles much more a stochastic experiment rather than reflecting the quality of the publications (Ike Antkare anyone?). Again, you have the effect of people wanting to cite some paper of a “high ranked” conference, because that is what people will cite in the future. And in order to be found more easily in the future via backwards citation searches, you’d rather cite publications you think will be cited more often in the future (cf.).

Talking about quality…

You have to trust the peer review of the conference or journal but you actually cannot because you don’t even know who the peers were. It’s good to have an informed opinion and it’s a good thing to be able to rely on an informed judgement. But it’s not good having to rely on that. If, for whatever reason, a peer fails to provide appropriate reviews, one should be able to make a decision oneself. Some studies have indeed shown that the peer review process is no better than flipping a coin. So there seems to be some need to review the peer review.

Once again to be clear: I don’t mind peer review. I think it’s good. Blindly publishing without ensuring that there is indeed an advancement of world’s knowledge wouldn’t be good. And peer review could be a tool to control that. But it doesn’t do it right now. I don’t have any concrete proposal. But I think if the reviews themselves and the reviewers were known, then we could make better decisions as to whether to “trust” a publication or not.

Another proposal is to not have “journals” as physical hard copies anymore. It is ~~2014~~2015, we have the Web, we have some cool technologies. But we don’t make use of any of that. Instead, we are maintaining the status from 20, or rather 200, years ago. We still subscribe to one-off bundles of printed and stapled paper. And we pay loads for that. And not only do we pay loads for receiving that, if you wanted to publish in one of those journals (or conferences), you have to pay, too. In fairness, it’s not only the printing and stapling that costs money, but the services around that. Things like proof reading (has anyone ever gotten a lectorate?), the peer review (has any peer ever gotten any reimbursement?), or the maintenance of an online database (why is it so damn hard to use any of these web databases?) are things we pay money for. I doubt that we need Journals in their current form. We probably do need entities (call them “publishers”), who in turn will need to earn some money, to make sure everything is going smoothly. But we don’t need print-and-forget style publishing. If we could add things like comments, annotations, links, reviews, supplementary material, a varying level of detail, to a paper, even after a few years or even decades, we could move to a “permanently peer reviewed” model. A publication is being reviewed all the time. Ideally by the general public. We could model our current workflow by delegating some form of trust to a group of people, say “reviewers of Journal X”, and only see what these people have vouched for. We could then selectively exclude people from that group of trustees, much like the web of trust. We could, if a paper makes an assumption which is falsified in the future, render some warning when opening the publication. We could decentralise the data such that everyone could build their own index, search mechanism, or interface.

On interfaces

Right now, if you wanted to, say, re-conduct the experiments done in published papers and share your results, you would have to create a publication (which is expected, but right now you would likely have to pay for that) and cite the papers whose results you are trying to reproduce. That’s okay. But if I then wanted to see when and how successful people tried to redo the experiments, I’d have to rely on the database I’m using to provide a reverse citation search and have the correct data (which, for some databases, seems to be the ability to do OCR on the PDF…). That’s not how things should work nowdays, right? We’d expect something more interactive, with tags, open data, something wikiesque. While the ability to reverse-search citations, to highlight some key references, or to link to a key contribution that followed a paper at hand would be nice indeed, we probably have to step back and make existing functionality somewhat usable. I’m not talking about advanced stuff like exporting search results in a standardised format or about deep linking to a result set from a query. That would need treatment after we’ve solved actually searching for multiple keywords, excluding some conferences or journals, or joining or intersecting queries. All that only works to some extent and it’s depressing that we cannot do anything about it, because we don’t have the relevant access or data. Don’t believe me? Well, you shouldn’t. But I’ll provide a table, probably in another post, showing what works with which database and what does not.

On experiments

As I was referring to reproducing results: It is pretty much impossible to reproduce any result, at least in my field, computer science. You don’t get the raw data, let alone the programs to run to get the results. You could argue that it is too complicated to maintain a program that can be run on any platform. Fair enough. I don’t have a solution. But the situation right now is not a good status quo. Right now you don’t get anything. So even if you had the very same setup as the authors of some publication, you would not be able to redo the experiments. It’s likely to be similar in other disciplines. I imagine that rocket scientists do experiments with self made devices or with some utterly expensive appliance (think LHC). Nobody will be able to reproduce the results, simply because there is just that one LHC out there… But… fortunately we have many digital things which are easy to archive and distribute. We, computer scientists, should make use of that. Why not require authors to submit a virtual appliance in some openly specified format? Obviously, source code would be nice, but even in academia there doesn’t seem to be a culture of sharing code freely, so I’m not even suggesting that.

Phew. After having criticised Academia and having made some half baked proposals I forgot what I actually wanted to do: Being a good academic (not caring about the public perception of “good” in terms of quantity of publications), and discuss a few things around the paper that we paid a couple of hundred dollars for to get published. But I leave that for another ~~rant~~ post.

In what ways do you think is Academia broken?

CHIS-ERA conference 2011 in Cork

While being in Ireland, I had the great opportunity of attending the CHIS-ERA strategic conference 2011 in Cork. Never heard of it? Neither have I. It’s a conference of European academic funding bodies to project and discuss future work and the direction of the work to be funded. Hence, it had many academics or industrial research people that talked about their vision for the next few years. If I got it correctly, the funding bodies wanted some input on their new “Call” which is their next big pile of money they throw at research.

The two broad topics were “Green ICT” and “From Data to Knowledge“. And both subjects were actually interesting. But due to the nature of the conference, many talks were quite high level and a bit too, say, visionary for my taste. But it had some technical talks which I think were displaced and given by poor Post-Docs that needed to have a presentation on their record to impress their supervisor or funding body.

CHIS-ERA Flower
However, for the Green IT part, almost all the speakers highlighted how important it was to aim for “Zero Power ICT”, because the energy consumption of electronic devices would shoot up as it did the last decade or so. But it hadn’t necessarily been much of problem, because Moore’s Law would save us a bit: We knew that in a couple of month, we could place the same logic onto half the chip which would then, according to the experts, use half the energy. However, that wouldn’t hold anymore in a decade or two, because we would reach a physical limit and we needed new solutions to the problem.

Some proposed to focus on specialised ICs that are very efficient or could be turned off, some others proposed to build probabilistic architectures because most of time a very correct result wouldn’t matter or to focus research on new materials like nanotubes and nanowires. The most interesting suggestion was to exploit very new non volatile memory technologies using spintronic elements. The weirdest approach was to save energy by eliminating routers on the Internet and have a non routing Internet. The same guy proposed to cache content on the provider as if it wasn’t done already by ISPs.

After the first day, we had a very nice trip to the old Jameson Distillery in Midleton. It started off with a movie telling us the story about Jameson coming to Ireland and making Whiskey. It didn’t forget to mention that Irish Whiskey was older and of course better than the Scottish and the tour around the old buildings were able to tell us what makes Irish Whiskey way better than the Scottish. Funnily enough, they didn’t tell us that the Jameson guy was actually Scottish 😉 I do have to admit that I like the Irish Whiskey though 🙂 The evening completed with a very nice and fancy meal in a nice Restaurant called Ballymaloe. I think I never dined with so many pieces of cutlery in front of me…

CHIST-ERA D2K visualisation
The second day was about “From Data to Knowledge” and unfortunately, I couldn’t attend every lecture so I probably missed the big trends. When I heard that Natural Language Processing and Automatic Speech Recognition were as advanced as being able to transcribe a spoken TV or radio news show with a 5% error rate, I was quite interested. Because in my world, I can’t even have the texts that I write corrected because I need to use ispell which doesn’t do well with markup or other stuff. Apparently, there is a big discrepancy between the bleeding edge of academic research and freely available tools 🙁 I hope we can close this gap first, before tackling the next simultaneous translation tool from Urdu to Lowgerman…