Behaviours developed in a state of scarcity may cease to be appropriate in a state of abundance. Our stone age ancestors struggled to get enough energy-rich food, so they acquired a taste for food with a strong energy hit. We inherited a greed for swee…
Following @carolecadwalla’s latest revelations about the misuse of personal data involving Facebook, she gets a response from Alex Stamos, Facebook’s Chief Security Officer.Hi, Carole. First off, I work on security, not strategy, and I agree that this …
Some people talk as if data protection and security must be fixed before May 2018 because of GDPR. Wrong. Data protection and security must be fixed now.Morrisons (2014)The High Court has just found Morrisons to be liable for a leak of employee data by…
Presentation given to the “GDPR Making it Real” workshop organized by DAMA UK and BCS DMSG, 12 June 2017.The presentation refers to two milestones. The second milestone is 25th May 2018, the date that companies will need to comply fully with the new da…
#PowerSwitch The relationship between the retailer and the customer can be beset by calculation on both sides. The retailer is trying to extract enough data about the customer to calculate the next best action, while the customer is trying to extract the best deal.
There is nothing new about customers comparing products and prices between neighbouring shops, and merchants selling similar goods can often be found in close proximity in order to attract more customers. (This is especially true for specialist and occasional purchases: in large cities, whole streets or districts may be associated with specific types of shop. London has Denmark Street for musical instruments, Hatton Garden for jewellery, Saville Row for made-to-measure suits, and so on.)
But nowadays the villain, apparently, is eCommerce. As a significant share of the retail business migrates from the high street to the Internet, many retailers are concerned about so-called showrooming. It may seem unfair that a customer can spend loads of time in the high street, wasting the time of the shop assistants and shop-soiling the goods, before purchasing the same goods online at a better price. To add insult to injury, some people not only practice showrooming, but then blog about how guilty it makes them feel.
The assumption here is that the Internet can generally undercut the High Street, and there are several reasons why this assumption is plausible.
- Internet businesses compete on price rather than service, so the prices must be good.
- An internet store can provide economies of scale – serving the whole country or region from a single warehouse, instead of needing an outlet in each town.
- An internet store can offer a much larger range of goods without increasing the cost of inventory – the so-called Long Tail phenomenon
- An internet store typically has lower overheads – cheaper premises and fewer staff
- An internet business may be run as a start-up, with less “dead wood”. So it is more agile and less bureaucratic.
However, there are some counterbalancing concerns.
- The economic and logistical costs of delivery and return can be significant, especially for low-ticket items. With clothing in particular, customers may order the same item in three different sizes, and then return the ones that don’t fit.
- Investors previously poured money into internet businesses, and the early strategic focus was on growth rather than profit. As internet business become more mature, investors will be looking to see some decent returns on their investment, and margins will be pushed up.
- And then there is differential pricing …
One of the key differences between traditional stores and online stores is in pricing. Although high street retailers often drop prices to clear stock – for example, supermarkets have elaborate relabelling systems to mark-down groceries before their sell-by date – they do not yet have sophisticated mechanisms for dynamic pricing. Whereas an online retailer can change the prices as often as it wishes, and therefore charge you whatever it thinks you will pay. According to Jerry Useem,
“The price of the headphones Google recommends may depend on how budget-conscious your web history shows you to be.”
I heard Ariel Ezrachi talking about this phenomenon at the PowerSwitch conference in Cambridge a few weeks ago. (I have not yet read his new book.)
“There is an assumption is that the internet is a blessing when it comes to competition. Endless choice. Ability to reduce costs to close to zero. etc … What you see online has very little to do with the ideas we have of market power, market dynamics, etc. everything is artificial. It looks like a regular market, with apples or fish. But because it’s all monitored, it’s not like that at all. What you see online is not a reflection of the market. You see “the Truman Show” — a reality designed just for you, a controlled ecosystem.” (via Laura James’s liveblog)
In his play Lady Windermere’s Fan, Wilde offered the following contrast between the cynic and the sentimentalist.
Lord Darlington: What cynics you fellows are!
Cecil Graham: What is a cynic?
Lord Darlington: A man who knows the price of everything and the value of nothing.
Cecil Graham: And a sentimentalist, my dear Darlington, is a man who sees an absurd value in everything, and doesn’t know the market price of any single thing.
According to one of the participants at the PowerSwitch conference, some eCommerce sites quote higher prices for Apple users, based on the idea that they are less price-sensitive and can afford to pay more. In other words, the cynical Internet regards Apple users as sentimentalists.
If there is an alternative to this calculative thinking, it comes down to reestablishing trust. Perhaps then retailers and consumers alike can avoid an artificial choice between cynicism and sentimentalism.
Emma Brockes, I found something I like in a store. Is it wrong to buy it online for less? (Guardian, 3 May 2017)
Ariel Ezrachi and Maurice Stucke, Virtual Competition: The Promise and Perils of the Algorithm-Driven Economy (Harvard University Press, 2016) – more links via publisher’s page
Laura James, Power Switch – Conference Report (31 March 2017)
Jerry Useem, How Online Shopping Makes Suckers of Us All (Atlantic, May 2017)
Price-bots can collude against consumers (Economist, 6 May 2017)
The Dilemma of Showrooming, (Daniels Fund Ethics Initiative, University of New Mexico)
Related posts: Online pricing practices to be regulated? (October 2009), Predictive Showrooming (December 2012), Showrooming and Multi-Sided Markets (December 2012), Showrooming in the Knowledge Economy (December 2012).
Last week was not a good one for the platform business. Uber continues to receive bad publicity on multiple fronts, as noted in my post on Uber’s Defeat Device and Denial of Service (March 2017). And on Tuesday, a fat-fingered system admin at AWS managed to take out a significant chunk of the largest platform on the planet, seriously degrading online retail in the Northern Virginia (US-EAST-1) Region. According to one estimate, performance at over half of the top internet retailers was hit by 20 percent or more, and some websites were completely down.
What have we learned from this? Yahoo Finance tells us not to worry.
“The good news: Amazon has addressed the issue, and is working to ensure nothing similar happens again. … Let’s just hope … that Amazon doesn’t experience any further issues in the near future.”
Other commentators are not so optimistic. For Computer Weekly, this incident
“highlights the risk of running critical systems in the public cloud. Even the most sophisticated cloud IT infrastructure is not infallible.”
So perhaps one lesson is not to trust platforms. Or at least not to practice wilful blindness when your chosen platform or cloud provider represents a single point of failure.
One of the myths of cloud, according to Aidan Finn,
“is that you get disaster recovery by default from your cloud vendor (such as Microsoft and Amazon). Everything in the cloud is a utility, and every utility has a price. If you want it, you need to pay for it and deploy it, and this includes a scenario in which a data center burns down and you need to recover. If you didn’t design in and deploy a disaster recovery solution, you’re as cooked as the servers in the smoky data center.”
Interestingly, Amazon itself was relatively unaffected by Tuesday’s problem. This may have been because they split their deployment across multiple geographical zones. However, as Brian Guy points out, there are significant costs involved in multi-region deployment, as well as data protection issues. He also notes that this question is not (yet) addressed by Amazon’s architectural guidelines for AWS users, known as the Well-Architected Framework.
Amazon recently added another pillar to the Well-Architected Framework, namely operational excellence. This includes such practices as performing operations with code: in other words, automating operations as much as possible. Did someone say Fat Finger?
Abel Avram, The AWS Well-Architected Framework Adds Operational Excellence (InfoQ, 25 Nov 2016)
Julie Bort, The massive AWS outage hurt 54 of the top 100 internet retailers — but not Amazon (Business Insider, 1 March 2017)
Aidan Finn, How to Avoid an AWS-Style Outage in Azure (Petri, 6 March 2017)
Brian Guy, Analysis: Rethinking cloud architecture after the outage of Amazon Web Services (GeekWire, 5 March 2017)
Daniel Howley, Why you should still trust Amazon Web Services even though it took down the internet (Yahoo Finance, 6 March 2017)
Chris Mellor, Tuesday’s AWS S3-izure exposes Amazon-sized internet bottleneck (The Register, 1 March 2017)
Shaun Nichols, Amazon S3-izure cause: Half the web vanished because an AWS bod fat-fingered a command (The Register, 2 March 2017)
Cliff Saran, AWS outage shows vulnerability of cloud disaster recovery (Computer Weekly, 6 March 2017)
When Complex Event Processing (CEP) emerged around ten years ago, one of the early applications was real-time risk management. In the financial sector, there was growing recognition for the need for real-time visibility – continuous calibration of positions – in order to keep pace with the emerging importance of algorithmic trading. This is now relatively well-established in banking and trading sectors; Chemitiganti argues that the insurance industry now faces similar requirements.
In 2008, Chris Martins, then Marketing Director for CEP firm Apama, suggested considering CEP as a prospective “dog whisperer” that can help manage the risk of the technology “dog” biting its master.
But “dog bites master” works in both directions. In the case of Eliot Spitzer, the dog that bit its master was the anti money-laundering software that he had used against others.
And in the case of algorithmic trading, it seems we can no longer be sure who is master – whether black swan events are the inevitable and emergent result of excessive complexity, or whether hostile agents are engaged in a black swan breeding programme. One of the first CEP insiders to raise this concern was John Bates, first as CTO at Apama and subsequently with Software AG. (He now works for a subsidiary of SAP.)
|from Dark Pools by Scott Patterson|
And in 2015, Bates wrote that “high-speed trading algorithms are an alluring target for cyber thieves”.
So if technology is capable of both generating unexpected events and amplifying hostile attacks, are we being naive to imagine we use the same technology to protect ourselves?
Perhaps, but I believe there are some productive lines of development, as I’ve discussed previously on this blog and elsewhere.
1. Organizational intelligence – not relying either on human intelligence alone or on artificial intelligence alone, but looking for establishing sociotechnical systems that allow people and algorithms to collaborate effectively.
2. Algorithmic biodiversity – maintaining multiple algorithms, developed by different teams using different datasets, in order to detect additional weak signals and generate “second opinions”.
John Bates, Algorithmic Terrorism (Apama, 4 August 2010). To Catch an Algo Thief (Huffington Post, 26 Feb 2015)
Vamsi Chemitiganti, Why the Insurance Industry Needs to Learn from Banking’s Risk Management Nightmares.. (10 September 2016)
Theo Hildyard, Pillar #6 of Market Surveillance 2.0: Known and unknown threats (Trading Mesh, 2 April 2015)
Neil Johnson et al, Financial black swans driven by ultrafast machine ecology (arXiv:1202.1448 [physics.soc-ph], 7 Feb 2012)
Chris Martins, CEP and Real-Time Risk – “The Dog Whisperer” (Apama, 21 March 2008)
Scott Patterson, Dark Pools – The Rise of A. I. Trading Machines and the Looming Threat to Wall Street (Random House, 2013). See review by David Leinweber, Are Algorithmic Monsters Threatening The Global Financial System? (Forbes, 11 July 2012)
Richard Veryard, Building Organizational Intelligence (LeanPub, 2012)
The Shelf-Life of Algorithms (October 2016)
@mrkwpalmer (TIBCO) invites us to take what he calls a Hyper-Darwinian approach to analytics. He observes that “many algorithms, once discovered, have a remarkably short shelf-life” and argues that one must be as good at “killing off weak or vanquished…
I recently went into a High Street branch of my bank and moved a bit of money between accounts. I could have done more, but I didn’t have any additional forms of identification with me.At the end, the cashier asked me for my nationality. British, as it…
I have been discussing Pay As You Drive (PAYD) insurance schemes on this blog for nearly ten years.
The simplest version of the concept varies your insurance premium according to the quantity of driving – Pay As How Much You Drive. But for obvious reasons, insurance companies are also interested in the quality of driving – Pay As How Well You Drive – and several companies now offer a discount for “safe” driving, based on avoiding events such as hard braking, sudden swerves, and speed violations.
Researchers at the University of Washington argue that each driver has a unique style of driving, including steering, acceleration and braking, which they call a “driver fingerprint”. They claim that drivers can be quickly and reliably identified from the braking event stream alone.
Bruce Schneier posted a brief summary of this research on his blog without further comment, but a range of comments were posted by his readers. Some expressed scepticism about the reliability of the algorithm, while others pointed out that driver behaviour varies according to context – people drive differently when they have their children in the car, or when they are driving home from the pub.
“Drunk me drives really differently too. Sober me doesn’t expect trees to get out of the way when I honk.”
Although the algorithm produced by the researchers may not allow for this kind of complexity, there is no reason in principle why a more sophisticated algorithm couldn’t allow for it. I have long argued that JOHN-SOBER and JOHN-DRUNK should be understood as two different identities, with recognizably different patterns of behaviour and risk. (See my post on Identity Differentiation.)
However, the researchers are primarily interested in the opportunities and threats created by the possibility of using the “driver fingerprint” as a reliable identification mechanism.
- Insurance companies and car rental companies could use “driver fingerprint” data to detect unauthorized drivers.
- When a driver denies being involved in an incident, “driver fingerprint” data could provide relevant evidence.
- The police could remotely identify the driver of a vehicle during an incident.
- “Driver fingerprint” data could be used to enforce safety regulations, such as the maximum number of hours driven by any driver in a given period.
While some of these use cases might be justifiable, the researchers outline various scenarios where this kind of “fingerprinting” would represent an unjustified invasion of privacy, observe how easy it is for a third party to obtain and abuse driver-related data, and call for a permission-based system for controlling data access between multiple devices and applications connected to the CAN bus within a vehicle. (CAN is a low-level protocol, and does not support any security features intrinsically.)
Miro Enev, Alex Takakuwa, Karl Koscher, and Tadayoshi Kohno, Automobile Driver Fingerprinting Proceedings on Privacy Enhancing Technologies; 2016 (1):34–51
Andy Greenberg, A Car’s Computer Can ‘Fingerprint’ You in Minutes Based on How You Drive (Wired, 25 May 2016)
Bruce Schneier, Identifying People from their Driving Patterns (30 May 2016)
See also John H.L. Hansen, Pinar Boyraz, Kazuya Takeda, Hüseyin Abut, Digital Signal Processing for In-Vehicle Systems and Safety. Springer Science and Business Media, 21 Dec 2011
Identity Differentiation (May 2006)
@dougnewdick points out the risk of a company becoming over-dependent on Google. His particular example is prompted by Google’s announcement that Google Reader will be discontinued.I have previously commented on the subject of Creeping Business Depende…
Michael Mainelli and Ian Harris have written a wide-ranging survey of economics, choice theory (game theory, psychology and ethics), systems theory, chaos theory, global warming and evolution. So what’s all that got to do with the price of fish?
One of the themes running through the book is that the price of fish bears no relation to the value of fish, especially if we are concerned about long-term value and the sustainability of fish stocks.
Oscar Wilde famously defined a cynic as one who knows the price of everything and the value of nothing. This definition has also been applied to accountants and economists. Michael and Ian are leaders of the Long Finance initiative, a movement within the City of London that aims to overcome this kind of short-term financial cynicism.
Michael and Ian describe the price of fish as a wicked problem – a problem that lacks easy definition as well as easy answers. “Sustaining the supply of edible fish is a wicked problem that presents global risks.” (p 301) And yet they suggest that the system might possibly sort itself out. “As fish run out and have to be sustainably fished, the historic underpricing of fish ceases.” (293)
But this is no time for naive optimism, and the system will undoubtedly need some intervention. “When the price is the same as the value, there are opportunities for sustainable financing. So far, price has not equaled value for fish. This is the biggest, wicked decision-making problem of all: knowing how to set a price that equals the value.” (p 295)
In other words, the problem is not just the alarming dwindling of fish stocks but the collective cynicism that not only led to this problem but also amplifies it and resists dealing with it effectively. The key word in the problem statement is the word “set” – even if a few clever people can agree what the right price of fish should be, the real challenge is to set this price into global trading and consumption systems.