Councils must use CSV for publishing spending

A guidance note published on the government's Data.gov.uk site on 10 September 2010 says that the spending data files must be published primarily in CSV (comma separated values) format, where each field of data is separated from the next by a comma. The note bans councils from using Microsoft Excel's XLS or portable document format (PDF) as the only formats for their spending data, although they can offer these as alternatives. "There is no reason why they should not do this, but this is not a substitute for the CSV files, and PDFs in particular do not allow their contents to be reused," reads the guidance note. This will require a change of approach from many of the councils already publishing spending data online. Research by the Guardian's Data blog indicates that of 53 local authorities it knows are publishing spending data online, 39% are publishing exclusively through PDF files, which cannot be opened and analysed with software such as spreadsheets. The coalition government has already said that all English councils must publish data on items of spending above £500 by January 2011. The guidance note adds that this should be published monthly no later than 30 days after the month ends, in individual files for each month and under a licence that allows open reuse, with the new Data.gov.uk licence recommended. It also requires councils to publish the name of an official, who will manage the release of the data and answer queries on it. "This will not only increase the skill levels within the council and improve relationships with reusers, but also reduce the necessity of going down formal routes (eg Freedom of Information requests)," says the guidance. It says that all spending over £500 including VAT should be included, but not salaries or compensation payments. It says that there are limited exceptions, such as payments to foster parents, where the payee's name can be redacted, but that the identity of individuals supplying councils, such as sole traders, should be normally be divulged. "There are no circumstances in local authorities where payment amounts and the record of the payment itself should be redacted as a whole," it adds. The guidance adds that as the data will not include prices, commercial confidentiality cannot be given as an excuse to avoid publication. The note encourages councils to publish all spending, not just that above £500, if possible. It also includes detailed guidance on the format to be used for the data. The Guardian Data blog research found that of the 53 councils – which it lists in full – 68% are Conservative controlled and 27% are in London and the south east.