Already our mobile apps and the Cyclescape project are open-source:

• iPhone (iOS) app
• Android app (on which we’ll be having a hack day soon – possibly in February)
• HTML5 mobile web app
• Cyclescape campaign group toolkit (plus deployment)

If you can help with any of those, or find the code useful, please do get involved!

The main site is not yet open-sourced, but work is in progress.

Our key aim here is to make it much easier to get other people involved. Over the last year we’ve had a number of enquiries from fellow cycling programmers about getting involved, but there have been very confusing aspects of the codebase – mainly very old code that had not been upgraded, and the setup has been difficult to say the least.

So our work at present is focussed on:

• Modularising the code much more, so that – for instance – people wanting to work on the interface don’t need to get a Python/C++ -based routing engine installed
• Setting up and securing a journey planner endpoint for the same reason (so that a repository download can work without the routing engine installed)
• Untangling key bits of code that have long been very problematic, even for us
• Splitting up classes further to reduce dependencies (currently there are over 150 classes and this is growing by the day – see screenshot below)
• Re-scripting various installation routines
• Making things more self-documenting
• Parametrising site-specific switches  in the code related to specific non-core sites that we are paid to run
• Migrating to Git and then Github (we currently use SVN)
• Deciding whether any sections should be closed-source

We’re doing these before open-sourcing because – realistically – we simply do not have the time resource at present to hand-hold people through an installation process that is currently more difficult than it should be. We know from our experience from our hack day back in 2010, that any barriers here will stifle involvement. This is also forcing us to deal with some of the nastier architectural design flaws that have constantly held back some work.

Much of the code is actually in a reasonably good state, however. It runs on Windows, Linux or Mac OSX without changes (though getting the core routing component running on Windows might be interesting!). It is mostly in PHP5 (error_reporting=2047), written in a traditional class-interaction format (not MVC), though there are many established design patterns such as front controller pattern, a central application resource, common data formats, common class extension principles, all database interactions through an abstraction layer, etc.

A few key issues for us, once open-sourced, are:

• Whether we should aim to push people towards running a single, central installation, and encourage people from abroad to contribute funds and code towards that
• The extent to which we can realistically give any support for other sites (given that we’re pushed as it is to get everything done)
• The use of the name ‘CycleStreets’ if we give any support for sites we don’t control
• Maintaining quality-control of hosted sites that we are paid to run
• Ensuring that the Photomap isn’t closely-coupled with the journey planner
• Addressing the lack of test coverage
• Internationalisation work

We’d welcome any feedback (in the comments below) on these points. For us the key problem is time – and funding if the system is to expand to other countries!

We don’t yet have a timescale, but the code cleanup is going quite fast now, and in fact is accelerating, as the removal of hacks is making things quicker to deal with.

Screenshot of our current classes, where most of the code is:

Rather than organising this along the lines of a bulk import, we are taking advantage of new technologies in Potlatch 2 and have commissioned Andy Allan, creator of OpenCycleMap, to develop new features to allow volunteers to collaborate on inspecting and merging the information into OSM.

This merging tool will also be of use for other external data that could be manually inspected and merged into OpenStreetMap.

Background

The DfT commissioned survey work in various cities around the UK for their Transport Direct product. In 2011 they released the results of the surveys as Open Data, in a complex GML format based on Ordnance Survey ITN data – unsuitable for use with OSM. However, in addition, they have funded work to convert the survey data to be based on OSM geometries suitable for incorporation. This has been done through brilliant work by Ralph of CCG.

The kinds of things surveyed include cycle routes, cycle parking, cycle lanes and their widths, surfaces widths and lighting conditions of cycleable paths, and so on. We are working to add support for these attributes into CycleStreets, so that routes are further improved.

In the UK wide areas of the cycling infrastructure have been mapped in OpenStreetMap, often more recently than the data from the DfT. Also, with the development of Vector Background layers in Potlatch 2, there was an opportunity to create an improved process for dealing with external datasets.

Further background information is available in blogs and on the mailing lists.

• Initial announcement of the project
• Update on the progress of the tool
• Talk-gb thread on the project

The demo

We’re pleased to announce that a demo is now available, and we’d like people to test it.

A demo is now available. It contains sample data for Nottingham and Cambridge, but it’s deliberately unable to save the data back to the main OSM server. When the final version of the data conversion is complete and available, this will be updated and fully able to work.

Two test areas are currently loaded:

• Cambridge
• Nottingham

How to use the merging tool

The merging process works as follows:

1. Click ‘Map style’ > ‘Wireframe’ to make things much easier to work with.
2. The background data is highlighted either orange (needs attention) or blue (already processed).
3. Click the background features to select them.
4. Ctrl+click (or cmd+click on a Mac keyboard) the relevant OSM feature (line) to see a side-by-side comparison.
5. Click on ‘Advanced’ in the left panel to see the merging controls.

Feel free to play around with this – the snapshot data is being reloaded from time to time as we get better imports, although we think we’re almost at a stage where the data conversion is fairly bug-free.

The merging tool is currently a beta and further improvements are planned. See the main Merging tool page on the OSM wiki.

The first screenshot shows the thick gray line (DfT data, as a background layer) highlighted. It shows the attributes it has:

The second screenshot shows what happens if we now control-click (or cmd+click on a Mac) on the OSM line – we now get a merging interface where we can accept/reject each attribute, and click the button at the end to accept all the changes:

Feedback on the data

We would really welcome feedback as to any errors you spot in the data conversion. The aim is that the data is pre-processed and snapped to the OSM geometry as effectively as possible, so that merging is merely a case of manual confirmation of each attribute according to your local knowledge.

Issues we have fed back so far on are:

• Alignment. The data was originally snapped to an OS Open Data, and has been geographically aligned via advanced GIS techniques to OSM. It’s already well over 90% matching and further improvements are being made.
• The issue of streets being broken up but having the same data. Our GIS contact plans to merge when the street name and data matches.

The software

A number of software components are used to make all this work

• Potlatch 2 is used as the editor, and can load data from both OSM and the DfT data. The splash pages and other resources are available on github
• Snapshot Server is used to serve the DfT data for each user, saving them from having to load the whole country at a time
• Some scripts are used for loading data in and out of the server. These use Osmosis to read/write between XML and Postgres.

License

The data is expected to be released under the Open Government License. We have been seeking an early letter of confirmation from the DfT on this and will update this page and the OSM Wiki accordingly. (The ITN-referenced dataset is released under the OGL already.)

Feedback

We’d really appreciate it if you could try out the beta and add comments below, or contact Andy Allan with any feedback you have. Did you figure out how to use the tool? Did you manage to merge some data? What doesn’t work? How could the tool be improved?

If you have local knowledge of the areas in question, it would be great to hear back from you on the datasets themselves – do they match reality? Are the tags appropriate?

As you can see, since our launch in March 2009, usage has grown very considerably this year.

We’ve planned almost 1.4 million cycle journeys, covering some 23 million km as the crow flies, equivalent to cycling to the moon 60 times!

It’s been incredibly heartening to see so many great comments from users of CycleStreets on how it’s helped them find better routes or start cycling.

Major projects this year

The year for us has been dominated by a number of funded major projects.

Work for the bicycle industry to create a Leisure Routing mode for the great cycle satnav app, Bike Hub (which uses our routing). Leisure routing is a new kind of routing, exclusive to the app, which enables you to plan a circular route lasting a specified time or distance. Read more about this great new addition to an already brilliant app.

Work with the Department for Transport to open up their cycling data. We’ve been helping the DfT with their very laudable objective to open up the cycling data they collected as part of their Transport Direct system. This detailed cycling attribute data has the potential to enhance the quality of cycle routing, by adding things like surface quality, cycle lane widths, missing pieces of data. We’ve funded some development work on a merging tool for the main OpenStreetMap editor, Potlatch 2, which we hope will be of wider benefit for other datasets also. The data is now almost ready for release following a lot excellent work by the DfT and its main contractor – stay tuned.

Cyclescape, a comprehensive online campaigning toolkit to assist cycle campaign groups around the UK. We’re able to create this thanks to funding from GeoVation. Earlier this year we won a Dragon’s Den -style contest to create a website that would build on our Photomap of 30,000 images, and provide a way for cycle campaign groups around the UK to work more effectively to solve cycling infastructure problems in their area. The website will be launched shortly, and we’re moving into beta with it after months of solid work. You can read more about this on the Cyclescape blog.

Work with Cycling Scotland to improve cycle journey planning in Scotland. We’ve been undertaking a range of projects to help motivate improved data collection and usage. This work will be finished shortly, and will see a journey planner on the Cycling Scotland website and a new guide to assist local communities with data collection for OpenStreetMap.

Hosted journey planners for London Cycling Campaign, West Sussex County Council, Bike Hub and others soon to launch! Read more about our offering of customised cycle journey planner sites and see the brochure for Local Authorities we created.

Other projects

As well as these funded projects, we’ve been working on several other areas of our core offering:

Continual improvements in route quality: We’re continually tweaking the routing engine to make the routes better and better, partly in response to the great feedback we get from users. In the summer we added partial support for surface quality data (something we’ll be building on soon), and earlier we undertook some work to make routes more ‘sticky’ to the Sustrans and Local Authority cycle networks where they exist.

Faster routing: The routing engine is now much faster than a year ago. Even when we’ve had a load spike, the hosting has barely flinched!

Mobile apps on all the major platforms have been released. As well as our free iPhone app, which has seen many improvements over the year, we’ve released a well-received and fully-featured Android app and a Mobile web site which works on Blackberry and other platforms. All of these are open source projects, and we encourage people to join our brilliant volunteers who have been working on the apps so far.

A lot of work on resilience: Simon has worked extremely hard since the summer to improve the resilience of the system in many ways: Backup, fallback, easier importing of data, scripting various administrative tasks and so on.

Points of Interest: We’ve got a new set of webpages and an API for points of interest (shops, cash machines, leisure locations, etc.). We hope to integrate this much more closely with other parts of our offerings in coming months.

Lots of other little changes: Addition of CO2 and calorie counts, an increase in the journey planning limit to 200 miles, little usability fixes, reworking of the Photomap pages, railway station codes in searches, new Photomap listing modes, promotional flyers you can order, clickable icons for the London Cycle Hire website, a ‘cycle to us’ link facility, an integrated map editor installed, etc.

Photo of the day: We hope you’ve enjoyed our Photo of the day feature, if you’ve been following us on Twitter!

What about next year?

The main areas we’re going to be working on are:

• Support for advanced routing data (work in progress) according to an advanced ruleset we’ve been considering
• Drag routing (waypoints in the interface)
• Expanding to other countries
• Open sourcing (the main barrier at the moment being lack of modularity in older parts of the code)
• Reworking the feedback system so that we can get lots of OpenStreetMap volunteers involved

OpenStreetMap – a solid foundation

CycleStreets wouldn’t have been possible without the brilliant OpenStreetMap (OSM) project, whose data we use. OSM has gone from strength to strength, with many areas of the country seeing far more detailed data than this time last year.

It’s a volunteer project which anyone can contribute to. So whether there’s a newly-added cycle lane near where you live, a bike shop has opened, or whether the surface of a path is unsuitable for cycling, you can contribute that information – just edit away.

If you’ve appreciated the work of the people who’ve created the map, please donate to OpenStreetMap to help keep it running fast.

The latest version of the Bike Hub cycle satnav app is 2.2; I’m currently testing 3.0, due for release soon. The new version will be packed with extra features, many of them suggested by app users.

I’m conflicted by all the additions. On the one hand I’ve always wanted to keep the app plain, simple, focussed. But users have said they want it to be feature rich.

2.2 – released at the end of October – was enriched with a GPX download feature so plotted routes could be emailed to a user for later use, perhaps in a route visualisation programme, such as Google Earth.

I’ve used this feature a couple of times but it’s not what I’d consider the app’s core function: this core function is to be a cycle-specific satnav. We have also been asked to include other ‘performance monitoring’ features but there are many other apps out there that do this well.

It’s a fine balancing act: making the app feature-rich but not so complex that it becomes hard to use or worse, buggy.

The app is a standard satnav in that it routes from A to B. Version 3.0 adds what could be a world exclusive and that’s A to A routing. This makes the app into a touring tool: arrive at a railway station with your bike, fire up the app to guide you on a three hour trip sticky to country pubs and off you go.

Part of this functionality was added to 2.2. You can navigate to and from Points of Interest: ATMs, places of worship, parks, castles and many other PoIs.

Version 3.0 uses this growing database as a tour suggester.

Also in version 3.0 – and a much requested feature – is dynamic route recalculation. Stray from the route and the app will suggest you make a u-turn. Stray a bit further and the app will create a new route, on the fly. This is standard on car satnavs, and it’s finally on Bike Hub version 3.0, thanks to app coding work from app developer Tinderhouse and routing tweaks by CycleStreets.

Version 3.0 also has tighter integration with CycleStreets, adding a box to input a CycleStreets journey number. This will be useful if you prefer to pre-plan your routes via desktop and then want to transfer the same route to your smartphone.

The most requested feature from users is added in version 3.0. This is map cacheing. Map tiles can be downloaded to a smartphone when in range of wifi or a good 3G signal. Users will still require a phone signal to call down the route from CycleStreets but maps – which can take a while to download in a poor signal area – can now be stored locally. (And deleted later, if wished).

I know I’ll use this feature for cycle tours in Northumberland.

The app has evolved greatly since launch and will continue to evolve, thanks to funding from the Bike Hub levy fund. If you have other features you’d like to see added, get in touch. editor at bikehub.co.uk

Carlton Reid

The new update to the CycleStreets Android app was released to the Android Marketplace this week. If you already have the app installed, it should update itself automatically. If you don’t have it installed, you can read about it in an earlier blog post and download it from the Android Market place here – for free.

This updates adds:

• New Points of Interest main menu option. Show points of interest, including cycle parking, pubs, cafes, post boxes, and London Cycle Hire points, on the route map. Choose the icon size from the settings menu.
• New main menu option Open route number. Want to load a journey you planned on the website? Just type in the journey number and off you go. Journey number is also shown on the itinerary tab.
• Saved routes can be renamed. Press and hold on a saved route to bring up the menu, then choose Rename.
• CO₂ and calorie savings: Itinerary now includes the estimated CO₂ saving from cycling rather than driving, along with the calories used on the journey.

Please do give a review of the app in the Android Marketplace! (Click on ‘User reviews’ then ‘Write a review’).

The Points of Interest menu option brings up the POI selection menu. You can choose as many different categories as you like, and they will be overlaid on the route map.

Tapping on a POI icon brings up a bubble with more information. Tap the icon again to dismiss the bubble, or tab bubble itself to start or end your route at the POI.

You can choose the size of the onscreen icons from the Settings screen.

The Points of Interest facility uses our POIs API available to mobile apps. It’s basically a convenience function replicating the same data in OpenStreetMap. We’ve focussed on practical location types in this app, as our main focus is on everyday, practical journeys.

Every route planned by CycleStreets is given a unique number. Journey number 1332563, for example, guides you from Euston Station to the Natural History Museum. Perhaps you planned a route on the website you now want to open in the app, or maybe someone emailed the route to you? Whereever that number came from the Open route number menu option lets you type it in open the route with the app.

If you’d like to help out with our app, perhaps to add new features, do grab the code and hack away!

The new primary server is needed to support expansion of the journey planner, to handle the growing amount of data we manage in the CycleStreets system and to provide increased reliability of our services,  as well as for our new project, Cyclescape. We now also have additional redundancy.

The switchover has turned out to be a rather more complex procedure than expected as there are quite a number of different components that are required to make CycleStreets all work smoothly. This recent switchover it has helped to tighten up a number of areas. This will make future switchovers easier to handle, and a major benefit has been a restructuring of our documentation.

We have tighted up how we handle file permissions and this means that developers no longer need sudo access to rollout code onto the live server. The structure we’ve got for that feels a lot cleaner and has improved the security of the system.

The hardest part of all this was the moment of switchover itself. As the system is live and being used to generate thousands of routes per day we aimed to have as little downtime as possible. The data on both systems was synchronised before turning off the live server and switching over the DNS. This was done in the small hours and because of careful prepartion was completed in about 15 minutes. When the new server became live it was impossible to tell that anything had changed. Only a few things were missed out in the switchover – the automatic tweeting of our photo-of-the-day, and the updating of a missed DNS record – but both of these are now fixed.

The system does feel more responsive now, and we’ve noticed that some of our scripts run twice as quick as previously.

We are very grateful to our hosts, Mythic Beasts, for their continued support and technical advice.

Funding for this upgrade has been partly helped by a grant from Cycling Scotland and from donations.

Read more on the Cyclescape website …

Andy Allan, who is perhaps best known for his excellent OpenCycleMap project, has been working for us to create a merging tool (within Potlatch 2) for helping to combine external datasets into OSM on a street-by-street basis. This tool will be useful not only for the DfT data that is becoming available soon, but also for other datasets.

The DfT data, which has mostly been collected by surveyors on bicycles, has the potential to significantly improve the quality of routing in some areas of England. We are well aware, however, that data collected by other agencies can undermine the work of OSM volunteers in the area if not handled sensitively, and so we've stressed that automated, bulk imports would not be accepted by the OSM community.

Instead, the approach taken is a method for OSM volunteers to inspect and merge the information on a street-by-street and attribute-by-attribute basis via the simplest and quickest means possible, using their local knowledge to enhance the end result.

Surface type, cycle lane widths and other data are amongst the attributes in the dataset, and these will shortly be supported in the routing engine.

The code for the beta release of the tool [included in this diff] has been reviewed and merged into the main Potlatch 2 codebase, and is already in the latest builds. We expect further changes to be made as we get feedback from initial testers.

OSMers should be able to try out the beta and the data soon as we have confirmation of the Open Government License for the data. Here are screenshots of the beta:

Cyclescape

We had over 70 suggestions (including some humorous ones), many of which were excellent ideas.

It became clear that many of the best ideas started with Cycle, and we liked the idea that the name would be co-branded with CycleStreets, i.e. Cycle-s-something.

We also thought it was particularly important to emphasise the geographical aspect of cycle campaigning. We thought that inventing a new word, Cyclescape would be a catchy and unique brand that emphasises the idea of improving the landscape for cycling across our towns and cities.

Having fixed on a shortlist, we began the process of trying to obtain domain names. This took a surprising amount of work as we wanted to ensure we got all the variants (.com/.net/.org) by way of brand protection.

Unlike three years ago when we bought the name CycleStreets, many Cycle+<something> names have now been taken and reserved by domain brokers. After approaching some domain holders, it was clear that some of the best names were going to be prohibitively expensive, in the thousands of pounds, which was well beyond an already-tight budget.

In the end we spent more than we wanted to acquiring the name, but we’re very pleased with it. We hope you like the name too!

Cyclescape.net is currently under development and we’ll be setting up a blog there shortly. @Cyclescape will also be the site’s Twitter identity, so do follow that also.

Andy and Andrew, our developers, have been making excellent progress. Many of the core concepts of the site are in place, so we’ll have screenshots of  the system at ‘alpha’ phase in the next few days. If you’re a coder, follow the code work in progress on Github.

Now we need a logo…

Now we need a logo! We like the idea of a curved section of city, with markers on and a bicycle going upwards. Anyone able to help with that?

CycleStreets has secured £27,000 of funding as a winner of Geovation – GeoVation is an Ordnance Survey initiative and forms part of the Ideas in Transit project with funding from the Technology Strategy Board and the Department for Transport.

Progress on our cycle campaigner toolkit is steaming (pedalling?) ahead! Coding work began on Monday, and will continue solidly for the next few months.

The development of the toolkit has been possible thanks to our GeoVation award, which secured us £27,000 of funding. GeoVation is an Ordnance Survey initiative and forms part of the Ideas in Transit project with funding from the Technology Strategy Board and the Department for Transport.

We're pleased to introduce our developer team, Andy Allan, Andrew France, plus myself (Martin Lucas-Smith) from CycleStreets as project manager. Andy and Andrew will be working in it on solidly for the next few months so that a large amount of development can be done.

We're also approaching various designers and design companies for the work on the design and information architecture aspects of the project.

Get involved in an exciting open source project!

The toolkit is to be developed as an open source project, with the code on Github. It will be written using the technologies of Ruby on Rails (v3.1), PostgreSQL, and jQuery.

Volunteers are needed to contribute to the code from the end of October. If you'd be interested, do let us know.

We're aiming to build a project team who will develop and look after the system from December onwards. By that stage, the grant-funded development work will mean that the system is already fully-featured and well coded so should be in a good state to add functionality to.

Timeline

We'll have:

• Screenshots of our 'alpha' available by 21^st October.
• A beta available to a few campaign groups from by 4^th November. If your group would like to be a 'guinea-pig', do let us know! We'll start to open it up to more groups in the month following.
• General availability of the site around 2^nd December, with bugfixes and additional features leading to the week before Christmas.

Stay tuned to this blog and our Twitter feed for updates.

We're also seeking an additional grant of around £10k to enable us to undertake solid coding on some of the more advanced features that groups have suggested, particularly do deal with more complex issues like cross-group sharing, which will increase the utility of the system considerably.

Designing the toolkit

We held our first developer meetings this week to develop the specification further and do some wireframing: