Category: openstack

Martyn Taylor and Steven Hardy have done an awesome job of demoing an Image Building Service for OpenStack:

I think this has huge potential. Imagine an OpenStack API to which you could send a request for a fresh image build of any OS, request specific packages, software or other content to be included and have that image be uploaded to Glance or Cinder once it’s built. The image gets built in the cloud on a Nova instance/VM and the cloud provider bills you for the compute and I/O resources needed to complete the build.

Laurence Miao did some great work recently to port cfg from optparse to argparse.

The only significant API impact was that we could no longer have CONF() return unparsed command line arguments.

We chose to offer two alternatives – firstly, support for positional arguments:

>>> CONF.register_cli_opt(MultiStrOpt(‘bar’, positional=True))
True >>> CONF([‘a’, ‘b’])
>>> CONF.bar
[‘a’, ‘b’]

and, secondly, integration with argparse’s sub-commands:

>>> def add_parsers(subparsers):
… list_action = subparsers.add_parser(‘list’)
… list_action.add_argument(‘id’)
…
>>> CONF.register_cli_opt(SubCommandOpt(‘action’, handler=add_parsers))
True
>>> CONF([‘list’, ’10’])
>>> CONF.action.name, CONF.action.id
(‘list’, ’10’)

After porting the CLIs in Nova (nova-manage), Glance (glance-control and glance-manage) and Keystone (keystone-manage) over to this sub-parsers stuff, it looks like it’s going to work out quite nicely.

We’re hoping to publish Nova, Glance, Keystone, Quantum, Cinder and Horizon 2012.2.1 next week (Nov 29).

The list of issues fixed so far can be seen here:

• https://launchpad.net/nova/+milestone/2012.2.1
• https://launchpad.net/glance/+milestone/2012.2.1
• https://launchpad.net/keystone/+milestone/2012.2.1
• https://launchpad.net/quantum/+milestone/2012.2.1
• https://launchpad.net/cinder/+milestone/2012.2.1
• https://launchpad.net/horizon/+milestone/2012.2.1

That’s roughly 80 bugs.

We’d appreciate anyone who could give the candidate tarballs a whirl:

• http://tarballs.openstack.org/nova/nova-stable-folsom.tar.gz
• http://tarballs.openstack.org/glance/glance-stable-folsom.tar.gz
• http://tarballs.openstack.org/keystone/keystone-stable-folsom.tar.gz
• http://tarballs.openstack.org/quantum/quantum-stable-folsom.tar.gz
• http://tarballs.openstack.org/cinder/cinder-stable-folsom.tar.gz
• http://tarballs.openstack.org/horizon/horizon-stable-folsom.tar.gz

We’ve also started drafting release notes here. Contributions to those release notes are very welcome.

The OpenStack Technical Committee and the OpenStack Foundation Board of Directors have pretty separate sets of responsibilities and can get on with their work independently. One exception to that is the inclusion of new projects in OpenStack.

In the coming weeks, members of the two bodies will decide how to clarify confusion around the term “core project” and what exactly happens projects who graduate through OpenStack’s Incubation process.

A thread on the openstack-dev mailing list is ongoing and is a great example of how a mailing list discussion can actually help to drive a rough consensus while still giving everyone an opportunity to express their views.

The TC is attempting to agree on a rough direction that represents the views of the TC before meeting with the Foundation Board. There are currently three distinct views. Firstly, this:

The concepts of “what is core” and “what is in OpenStack” have been conflated until now. The TC cares far more about the process for new projects to be included in the coordinated release than it cares about which projects are required to be used by providers in order to access the trademark.

We would like to take an inclusive but measured approach to accepting new OpenStack projects. We should evaluate any given proposed project on a well defined set of criteria like whether it embraces our values and processes, is useful to OpenStack users, well integrated with other projects and represents a sensible broadening of the scope of OpenStack.

We see Incubation as a trial period where promising projects have the opportunity to demonstrate their suitability for inclusion in our coordinated releases.

We see the term “Core OpenStack Project” in section 4.1.b of the bylaws as being solely related to trademark guidelines. The Foundation should simply maintain a list of projects required for trademark usage. We would be happy for that list to be called “Core Projects” or for a new name to be chosen to describe that list.

Secondly, Anne Gentle’s variation which I’d summarize as allowing two groups of projects be accepted into OpenStack – “nuclear” projects which are the current group of “core” service projects and “core” projects which are everything else, for example Horizon, Ceilometer or Heat.

Thirdly, John Dickinson’s variation which I’d summarize as only accepting projects into OpenStack which “solve IaaS problems” or support those projects in some way.

The way I’m thinking of these three approaches is how we want the project to treat proposals for new projects: “inclusive”, “inclusive but two-tier” or “exclusive”.

As I did for Essex, I’ve used Jonathan Corbet’s gitdm to do some analysis of the commits to the core projects during Folsom.

Here’s the top 20 committers across Nova, Glance, Swift, Keystone, Horizon, Quantum and Cinder:


Processed 3110 csets from 291 developers
132 employers found
A total of 350046 lines added, 275491 removed (delta 74555)

Developers with the most changesets
Russell Bryant 160 (5.1%)
Brian Waldon 160 (5.1%)
Dan Prince 154 (5.0%)
Gabriel Hurley 124 (4.0%)
Mark McLoughlin 118 (3.8%)
Johannes Erdfelt 113 (3.6%)
Vishvananda Ishaya 92 (3.0%)
Joe Gordon 85 (2.7%)
Michael Still 71 (2.3%)
Eoghan Glynn 70 (2.3%)
Rick Harris 59 (1.9%)
Gary Kotton 58 (1.9%)
Dolph Mathews 55 (1.8%)
Yun Mao 50 (1.6%)
John Griffith 45 (1.4%)
Daniel P. Berrange 45 (1.4%)
Dan Wendlandt 43 (1.4%)
gholt 40 (1.3%)
Rongze Zhu 39 (1.3%)
Alex Meade 39 (1.3%)
Covers 52.090032% of changesets


Congrats and thanks to everyone involved in this release!

There are more raw stats here showing stats for each project individually and also statistics for gerrit reviewers and launchpad bug fixers.

Of course, this is nowhere near as polished as Bitergia’s awesome report with pretty graphs and detailed analysis.

My mate Derek was giving me grief about not testing his OpenStack deployment in our lab at Red Hat. Friday seemed like a good day to give it a shot for a few minutes.

First problem – I’m one of the weird people at Red Hat who eschews the VPN in favour of SSH tunnels. At first, I figured I’d tunnel directly to the various OpenStack API services but that didn’t work because the endpoint URLs returned by keystone obviously wouldn’t point to my tunnelled connections.

Ok, let’s just use a HTTP proxy, that should be fine. But no, not on yak shaving day. For some reason, I was getting 403 Forbidden errors.

To cut a long story short, it turns out:

• httplib2 always uses HTTP CONNECT tunneling rather than just sending the requests directly to the proxy
• squid by default and, indeed, our corporate proxy defaults to rejecting CONNECT for ports other than 443
• The recently released httplib2 0.7.5 has a PROXY_TYPE_HTTP_NO_TUNNEL which only uses CONNECT tunnelling for port 443, but it doesn’t use this type when you configure your proxy via http_proxy in the environment

Not content with shaving the yak once, I shaved her thrice:

• An upstream patch to use HTTP_NO_TUNNEL when using http_proxy env
• A request to get the patch into Fedora
• A patch to work around it in python-novaclient

One other troubling conclusion is that if you’re exposing the services over HTTPS, you really should use port 443 for everything or clients won’t be able to connect over many proxies.

I just wrote down a few pieces advice for someone submitting a large feature patch to Nova, so I thought I’d re-post it here:

• Think about what it is like to be a nova-core reviewer looking at a list of 40 to 60 reviews and having maybe 2 hours today to do reviews. Think about how to make it more likely that such a reviewer will choose your code to review.
• Small patches are easier to consume. The smaller you make the patch, the more likely it is that it will get reviewed quickly.
• Break your changes into a series of small, self-contained changes.
• The earlier in the release cycle you can begin submitting some of your changes the better. Don’t wait until all of your changes are finished before submitting.
• Do as much of your design discussion in the open, preferably on the openstack-dev mailing list. If you have discussions on the phone or IRC, try and post a summary of those discussions to the
  mailing list.
• Holding a design summit session to discuss your changes in advance is a great idea, but don’t assume that everybody who may have an opinion on your changes is present. Also, bear in mind that someone’s quick opinion offered at a design summit session may be very different from their considered opinion after reviewing the code in detail.
• Finally, try and participate in the project beyond just making the changes you need. Review other peoples’ changes in gerrit and offer your opinions, participate in design discussions on the mailing lists, fix bugs you come across, triage bug reports, etc. etc. All of this will allow other developers to get to know you, trust your judgement and review your changes more quickly. You will also learn more by interacting with other developers.

The OpenStack Foundation Board election has begun with 39 excellent candidates for only 8 seats!

Each candidate was asked to answer a number of questions to give an insight into why they are running for the Board. My answers are below.

If you’d like to know some more about my involvement in OpenStack, Rackspace recently posted a nice interview with me as part of their “How I Contribute To OpenStack” series.

What is your relationship to OpenStack, and why is its success important to you and/or your company? What would you say is your biggest contribution to OpenStack’s success to date?

I’ve been involved with OpenStack for just over a year now. I started out packaging OpenStack for Fedora, but have concentrated mostly on contributing to the core projects in various ways since then.

I’m a member of the nova-core team and spend quite a bit of time reviewing proposed changes in gerrit. I’m also the openstack-common PTL and have been working to kickstart this effort to create a shared library for all OpenStack projects. My role as PTL recently means that I’m now a PPB/TC member, but I haven’t contributed much there yet. Finally, I co-ordinate the efforts of the stable-maint and release teams to maintain a stable branch for OpenStack’s core projects and publish regular bugfix releases.

It’s thanks to Red Hat that I have time to make these contributions to the project and I am the technical lead of Red Hat’s OpenStack team.

Red Hat’s mission statement is all about being a catalyst in communities and, while credit must go to others for initially catalysing the OpenStack community, we recognize the massive strength of OpenStack’s diverse community. This strength is the foundation that will ensure that OpenStack continues to grow, continues to improve and will be around for a long time to win. Red Hat is doing everything it can to help make that happen.

Describe your experience with other non profits or serving as a board member. How does your experience prepare you for the role of a board member?

I’ve been heavily involved in various open-source projects for over a decade now. Observing and pondering the role of non-profit organizations in open-source projects has prepared me well to participate in the OpenStack Foundation and its efforts to determine its own particular role.

Indeed, I participated significantly in the early discussions leading up to the formation of the Foundation and even proposed a strawman Foundation structure based on my experiences with other similar organizations.

What do you see as the Board’s role in OpenStack’s success?

More importantly, I think, is the role the Foundation will play in OpenStack’s success. I’m a big fan of the Foundation’s simple mission statement – to “protect, empower and promote” OpenStack. It perfectly encapsulates the role of the Foundation.

The “protecting” and “promoting” elements of the mission are hugely important, but what I find myself wondering most about is how the Foundation can “empower” the project. The Foundation can certainly use some of its budget to fill gaps in the resources which are available to the project. But, to me, “empowering” the project is mostly about re-inforcing the understanding that the project’s future and direction lies in the hand of its meritocractic community of contributors (in the broadest possible sense) and not the Foundation itself.

I think that if the Board delivers on the promise of a Foundation which “protects, empowers and promotes”, we will find ourselves in the enviable situation of a project with a large and diverse community of contributors supported by a well-financed organization acting at all times in the interests of that community.

What do you think the top priority of the Board should be during the OpenStack Foundation’s first year?

I think it’s the transitioning of responsibilities from Rackspace to the Foundation will take much of the focus of the Foundation and the Board in the coming year. The Board will need to appoint an Executive Director and, in turn, oversee the Executive Director’s work to transition the likes of community management, release management and event co-ordination to employees of the Foundation.

A huge part of OpenStack’s success to date is that functions such as these have been executed extremely well under Rackspace’s management. The transition to a Foundation is a significant challenge in its own right and we cannot afford any missteps here. As part of this transition, I look forward to more open discussion leading up to decisions made by the Foundation.

I also expect the Board to quickly establish the User Committee and Legal Affairs Committee. The User Committee will have a hugely important role to help advocate specific needs of operators and users. The Legal Affairs Committee will be needed to help us make progress on reforming our trademark and contributor agreements.

Finally, the Board will have its work cut out finding its feet as the official voice of OpenStack. The Board will be expected to speak on behalf of us all and, as such, will need to work hard to build and reflect consensus within the project.

So, you want to play with some sqlalchemy queries against the Nova DB in an interactive python console?

$> sudo nova-manage shell python
...
>>> from nova.db.sqlalchemy import api
>>> from nova.db.sqlalchemy import models
>>> from nova import context
>>> ctxt = context.get_admin_context()
>>> vars(api.model_query(ctxt, models.Service).all()[0])
{'binary': u'nova-compute', ..., 'topic': u'compute', 'host': u'f16', 'disabled': False, 'deleted_at': None, 'id': 1L}