Personal Travel Plan Project in Harringay

Replies to This Discussion

Permalink Reply by Dan on May 22, 2015 at 12:37

Hi Emina

As a behavioural scientist and resident - I must query this spend of money, particularly when public services are under such stress. Smarter travel campaigns are an expensive collection of  beacons, entrenched in the – “if we can change people’s attitudes, then we can change their behaviour” camp. This is wrong. There is no causal (i.e. that variable a = outcome b) evidence at all to suggest they work. They are a way that councils can tell people they care about travel, which costs thousands upon thousands of pounds.

Let me explain why.

Firstly I get the underlying care and intention to improve transport systems in the borough. Understanding travellers’ preferences is a fundamental part of this. This is where we part.

I take issue with your use of the words 'behaviour' and 'change' in this context.

Without going into a lecture about what behaviour is, let’s just say in brief terms - it's what we do. Not what we say.

What evidence shows that surveys (aka stated preferences) are linked to behaviour (revealed preferences)? Do we really say what we do? Surveys show very very little. It is a dated way of addressing a problem. Sure surveys can illicit preferred responses (through clever framing of questions) and can with good design give insights into subjective well-being and other subjective preferences.

But stated preferences are heavily affected by psychological biases - your process of sample collection further muddies this through selection biases (aka hawthorne effects - people want to look good when they're being recorded).

So my first questions to Haringey is: 1) How much does this programme of surveys cost? and 2) what studies using a good evidence base show cost-effectiveness? I re-iterate that I get the good intentions and that they council want to do well. I am interested as to how success is demonstrated.

My second issue is your use of the word ‘change’. In order to 'change' something, a variable must be demonstrated versus a counterfactual or in this case a control. This is why all medicines use randomised control trials. This is the only way to demonstrate what works with enough confidence. Please see John List’s excellent work on field experiments which explains this very well.

So what is the currency of smarter travel? The number of people who say they do x?

The whole design of the programme is entrenched in a default position which cries out for innovation and behavioural insights. Solutions to follow (which are much more cost effective!)…

Do neighbourhoods matter to outcomes? Which classroom interventions improve educational attainment? How should we raise money to provide important and valued public goods? Do energy prices affect energy demand? How can we motivate people to become healthier, greener, and more cooperative? These are some of the most challenging questions policy-makers face. Academics have been trying to understand and uncover these important relationships for decades.

Many of the empirical tools available to answer these questions do not allow causal relationships to be detected. Field experiments represent a relatively new methodological approach capable of measuring the causal links between variables. By overlaying carefully designed experimental treatments on real people performing tasks common to their daily lives, we can answer interesting and policy-relevant questions that were previously intractable. Manipulation of market environments allow us to uncover the hidden motivations behind behaviour more generally. A central tenet of field experiments in the policy world is that governments should understand the actual behavioural responses of their citizens to changes in policies or interventions.

I have presented at Haringey council (sept 2014) at the request of senior management with one of the best behavioural scientists around - Dr Robert Metcalfe form the university of Chicago (who wrote the 2 paras above). This was attended by a large number of senior management. 

The smarter travel programme is well-intentioned but far from smart/cost effective. I'd be happy to meet you in person to talk about how behaviour change can really help the council. I have delivered a number of proven and peer-reviewed projects that have done this in local authorities. 

If you want to take one thing away from this message - please gather revealed preference data. How about council staff install an app (i.e. ‘wecycle’) onto their phones so their commute can be logged automatically (no threat of privacy – check it out)? Residents can do the same to help build a picture of how people are moving around the borough. That’s a lot closer to measuring what's behaviour. Then we can think about interventions and delivering versus a control.

I would finally like to add, it is wonderful to see 'nettles eyes' along the passage to try and stop dog mess. Although this requires much more input, project design and ultimately funding to show causality, this is an application of behavioural science and whoever does this needs commending.

Emina – I ask you in these times where central government will make huge cuts to some of our most vulnerable to question the spend on such unproven and expensive opinion collections as ‘smarter travel surveys’. I am not sure what this has to do with behaviour.

Thanks for reading

Daniel

 

 

• ▶ Reply

Permalink Reply by Alan Stanton on May 22, 2015 at 13:29

Daniel, I'd be interested to learn your views on a wide variety of Council initiatives which are often well-intentioned but far from showing any rigour. Or which lack even basic requirements such as clarity of aims; record keeping; independent evaluation etc. etc.

Many people are now having to resort to Freedom of Information Act requests to prise even basic information out of Secret Haringey. Yet large sums of scarce funding are now spent on projects across the Council - for example to make supposed savings and efficiencies; or to "regenerate" parts of the borough. We seem to give preferential treatment, and to hand out cash to favoured businesses as if there was a money tree behind the Civic Centre.

The saddest thing is that most of the time it seems likely that neither councillors nor senior officers actually know what works or doesn't and why. Nor it seems, are Haringey's key decision-makers even mildly curious. Provided they can announce a new upbeat project.

Which doesn't prevent them being quick to claim "success" and disown or ignore failure.  While, in the current culture of fear, anyone who thinks they do know what works or has criticisms of what doesn't, is well-advised to keep quiet. Though as the more experienced and competent professional staff leave and take their learning with them, I expect their temporary consultant and "interim" replacements will be careful to say only what their employers want to hear.

Out of curiosity, were you and your U.S. colleague making a sales pitch for funding? Nothing whatever wrong with that, of course.

• ▶ Reply

Permalink Reply by Dan on May 22, 2015 at 14:07

Hi Alan, thanks for your advice. This area is very new, so although entrenched in academia for many years, this is new to the 'field'. The benefits to the public sector are huge. You are right rigour is key. 

To answer your question - I live in haringey and want to use my skills and proven scientific methods to make my home a better place. I thought this meant speaking to the council about how behavioural science can be applied to thinking. We presented for free. 

A project I did in a neighbouring borough had an ROI of c3000% http://cep.lse.ac.uk/pubs/download/dp1222.pdf  Our team were behind the famous HMRC work that brought in revenue of over £13m just by changing a sentence on a letter. 

http://karlan.yale.edu/fieldexperiments/papers/00391.pdf

It wasn't a sales pitch - but there's no doubt that using behavioural science and RCTs to evaluate policies is extremely cost effective. I wont lie - I'd love to work with LBH as it's my home. But looking beyond this, the sooner policy makers get this the better.

Linking strong evaluation with excellent behavioural insights is a very powerful tool for policy makers. 

As you asked, a project we are trying to do with LBH and LSE is in fact applied as fully-funded. This is a project with homelessness and financial education. 

I am working with a lot with personal travel right now, commuting behaviour and preferences is a personal favourite.

I also think that failure can be even stronger as a positive lesson. 

This smarter travel is not behaviour though. It's attitude at best. 

• ▶ Reply

Permalink Reply by Alison P on May 22, 2015 at 14:28

Hi Daniel 

Thanks for a very interesting post. I agree with a great deal of what you've said, particularly about the importance of making sure that measuring the success of any project like this has to grapple with the difference between actual and stated behaviour (I say this as someone who has worked for years in the survey research industry!). 

On your point about the stickers with eyes on them in the passage - they have been put there by Ant (who is on this site and is part of a group called Friends of Harringay Passage) as part of his interest in looking at what we could do to reduce the amount of dog mess in the passage. There is a discussion about it at the link below - but I'm not sure where he is at on evaluating its success. 

http://www.harringayonline.com/group/friends-of-harringay-pasage/fo...

• ▶ Reply

Permalink Reply by Dan on May 22, 2015 at 14:34

Thanks Alison

This would make a great study with a dash of randomisation, a larger sample and of course - a poo surveyor!! But it would be possible - I'm sure with the right design we could get a university on board. Huge policy implications. 

It's better than this - http://www.standard.co.uk/news/london/dna-tests-on-dog-poo-to-trace... !

It is great to see though. 

• ▶ Reply