links

• Preview of Storm: The Hadoop of Realtime Processing – BackType Technology

  Curious to see what happens with Storm, post-Twitter acquisition of BackType

  “There are three broad use cases for Storm:

  1. Stream processing: This is the traditional realtime processing use case: process messages and update a variety of databases.

  2. Continuous computation: Storm can be used to do a continuous computation and stream out the results as they’re computed. For example, we used Storm the other day to compute trending users on Twitter off of the Twitter firehose. Every second, Storm streams out the 50 users with the most retweets in the last few minutes with perfect accuracy. We stream this information directly into a webpage which visualizes and animates the trending users in realtime.

  3. Distributed RPC: Distributed RPC is perhaps the most unexpected and most compelling use case for Storm. There are a lot of queries that are both hard to precompute and too intense to compute on the fly on a single machine.”

  tags: storm backtype oss real-time

• The challenges of streaming real-time data – O’Reilly Radar

  “You would be shocked at the ratio of engineers who can’t build event-driven, asynchronous data processing applications, to those who can, yet this is a big part of this space.”

  “…Jud Valeski: “Big data” as we talk about it today has been slayed by lots of cool abstractions (e.g. Hadoop) that fit nicely into the way we think about the stack we all know and love. “Big streams,” on the other hand, challenge the parallelization primitives folks have been solving for “big data.” There’s very little overlap, unfortunately. So, on the software solution side, better and more widely used frameworks are needed…”

  tags: real-time streaming event-driven

Posted from Diigo. The rest of my favorite links are here.

Related posts:

1. Link Collection – July 3, 2011
2. Link Collection (weekly)
3. Link Collection (weekly)

• A Simple Answer to “What Is Billing?” | Doug Newdick’s Blog

  “In my last blog post I talked about what #sexybilling was, or perhaps more accurately , what it might look like. It became evident from the comments though that I hadn’t really made it clear what I thought billing was, and so that sort of complicated the story. If we don’t know what billing is, how are we going to know what sexy billing is?”

  tags: billing

• How to get Ahead in Enterprise Architecture by Drawing Boxes – Simplicable

  “Last week I spoke with the chief architect of a large international retail bank. This bank was able to reduce their application map from 160+ systems down to 5.

  How (you may ask)? The answer is simple — by drawing boxes around them.”

  tags: entarch

• The real (potential) impact of SAP HANA

  “Much has been written about SAP HANA. The technology has been variously described as “transformative” and “wacko.” Well, which is it?”

  “In-memory databases take advantage of two hardware trends: a significant reduction in the cost of RAM, and a significant increase in the amount of addressable memory in today’s computers. It is possible, and economically feasible, to put an entire database in memory, for fast data management and query. Using columnar or other compression approaches, even larger data sets can be loaded entirely into main memory. With high-speed access to memory-resident data, more users can be supported on a single machine. Also, with an in-memory database, both transactional and decision-support queries can be supported on a single machine, meaning that there can be zero latency between data appearing in the system, and that data being available to decision-support applications; in a traditional set-up where data resides in the operational store, and then is extracted into a data warehouse for reporting and analysis, there is always a lag between data capture and its availability for data analysis.”

  tags: SAP HANA in-memory dbms active-information

Posted from Diigo. The rest of my favorite links are here.

Related posts:

1. Link Collection (weekly)
2. Link Collection (weekly)
3. Link Collection (weekly)

• Obama IT Czar Leaves D.C. for Harvard – Technorati IT

  Setting direction is easy. Executing transformation is hard.

  “A massive task and challenge awaited Mr. Kundra as he sought to change the direction of the government’s $80 billion annual Information Technology budget. In December 2010 the changes in direction were outlined in his 25 point plan to reform government IT.

  Reports of his success levels vary, but no doubt Kundra stirred the pot and set a direction. Changing the direction of anything at the Federal government is not for the faint or impatient heart.”

  tags: execution transformation IT government

• High Scalability – High Scalability – 35+ Use Cases for Choosing Your Next NoSQL Database

  “We’ve asked What The Heck Are You Actually Using NoSQL For?. We’ve asked 101 Questions To Ask When Considering A NoSQL Database. We’ve even had a webinar What Should I Do? Choosing SQL, NoSQL or Both for Scalable Web Applications.

  Now we get to the point of considering use cases and which systems might be appropriate for those use cases.”

  tags: scalability nosql

Posted from Diigo. The rest of my favorite links are here.

Related posts:

1. June 12 Link Collection
2. Link Collection (weekly)
3. Link Collection (weekly)

• VMware: Rethink IT: The future of cloud and NYSE Euronext’s capital markets community platform

  “NYSE Technologies (a unit of NYSE Euronext) announced their new “community platform” cloud computing service today, running on vSphere and vCloud Director from VMware. The target customers for the service are capital markets organizations such as hedge funds and the trading departments of banks, with initial customers including a unit of Goldman Sachs and the hedge fund Millenium Partners.

  …

  NYSE’s service is innovative in several ways. NYSE Euronext is primarily known for being a financial exchange as well as a provider of market data, rather than as a cloud provider. So why start offering cloud computing? NYSE saw how it could significantly simplify and improve its customers’ competitiveness in capital markets by providing an integrated service that combined on-demand computing with access to the market (the exchanges), a low-latency secure network and instant access to data feeds. In a reversal of traditional approaches to IT, computing capacity is literally coming to the market and the data — rather than the data and market being piped to the computers.”

  tags: nyse cloudcomputing capitalmarkets

• 5 Technologies That Will Shape the Web – IEEE Spectrum

  “Today the Web is going through another reinvention, morphing into a place where our social interactions are ever more important. And the main force behind this phenomenon is, of course, Facebook, led by Zuckerberg, now a 27-year-old billionaire.

  So where will the Web go next? We asked two dozen analysts, engineers, and executives to describe what technologies they think will shape our online experiences in the next several years. Their predictions could easily fill this entire issue, but we distilled their wisdom into a more palatable list of five key technologies that our sources mentioned most frequently.

  We also asked six of the experts to tell us what these technologies mean for today’s dueling titans, Google and Facebook. What challenges do they face? Who’s got an advantage?”

  tags: tech trends

• EA: To infinity and beyond! « Adam Deane

  Adam Deane employs the Toy Story characters as Enterprise Architect and (requisite) cast of doubters:

  “Buzz: All this is going to change. I am Buzz Lightyear; I am the new generation of Enterprise Architects. No longer IT oriented. I’m business oriented. I come in peace.

  Woody: How are you going to make a difference?

  Buzz: I will create business growth instead of trying to find ways to save money. I will generate revenue. I will speed up business cycles. I will make myself indispensable to the organisation.”

  tags: entarch humor

Posted from Diigo. The rest of my favorite links are here.

Related posts:

1. Link Collection (weekly)
2. Link Collection (weekly)
3. Link Collection (weekly)

• The Amazon.com 2010 Shareholder Letter Focusses on Technology – All Things Distributed

  SOA, data management advances, cloud computing and more, in use for Amazon’s core business:

  “Our technologies are almost exclusively implemented as services: bits of logic that encapsulate the data they operate on and provide hardened interfaces as the only way to access their functionality. This approach reduces side effects and allows services to evolve at their own pace without impacting the other components of the overall system. Service-oriented architecture — or SOA — is the fundamental building abstraction for Amazon technologies. Thanks to a thoughtful and far-sighted team of engineers and architects, this approach was applied at Amazon long before SOA became a buzzword in the industry. Our e-commerce platform is composed of a federation of hundreds of software services that work in concert to deliver functionality ranging from recommendations to order fulfillment to inventory tracking. For example, to construct a product detail page for a customer visiting Amazon.com, our software calls on between 200 and 300 services to present a highly personalized experience for that customer.”

  “The storage systems we’ve pioneered demonstrate extreme scalability while maintaining tight control over performance, availability, and cost. To achieve their ultra-scale properties these systems take a novel approach to data update management: by relaxing the synchronization requirements of updates that need to be disseminated to large numbers of replicas, these systems are able to survive under the harshest performance and availability conditions. These implementations are based on the concept of eventual consistency.”

  tags: amazon soa cloud information_dissemination

• Drenched by the Cloud | CFOworld

  The cloud is not a substitute for planning, architecture, risk management and common sense. Know what you are doing. The finger pointing here should not be at Amazon, but the decision-makers who naively delegated all control to someone else.

  “Already Downing and his team have taken significant steps to ensure they are never caught like this again. “We’ve had a series of meetings here internally to review all points of failure in our cloud strategy. We’re digging deeper to find out where data is hosted and what the backup plans are for that data,” he says. He adds that he’ll be hosting the main database at an additional cloud service for redundancy – a cost he calls blatantly necessary in light of this situation.”

  tags: amazon cloud management

• Silicon Valley and the technology industry: The new tech bubble | The Economist

  Is there a bubble? Or is the bubble conversation a bubble?

  “With luck the latest web bubble will do less damage than its predecessor. In the 1990s internet euphoria caused a dramatic inflation in the price of telecoms firms, which were creating the infrastructure for the web. When internet firms’ share prices plummeted, telecoms investors suffered too. So far, there has been no sign of such a spillover effect this time around. But the globalisation of the internet industry means that many more people could be tempted to dabble in web stocks in the current boom, adding to the pain of the bust.

  When will that be? This paper warned about both the last internet bubble and the American property bubble long before they burst. Irrational exuberance rarely gives way to rational scepticism quickly. So some bets on start-ups now will pay off. But investors should take a great deal of care when it comes to picking firms to back: they cannot just rely on somebody else paying even more later. And they might want to put another bumper sticker on their cars: “Thanks, God. Now give me the wisdom to sell before it’s too late.””

  tags: technology investing

• The Sensors Are Coming! – NYTimes.com

  ”Sensors will be everywhere in the next few years and will be able to help people become more conscious of the environment and our own health,” explained Mr. Vigna. “Your socks, shoes, glasses and even your garbage can will have sensors inside designed to help you manage everything from your effects on the environment to your health.”

  tags: instrumentation internet-of-things active-information

Posted from Diigo. The rest of my favorite links are here.

Related posts:

1. Link Collection (weekly)
2. Link Collection (weekly)
3. Link Collection (weekly)

Bye, Bye My Clustered AMIs…A Cloud Tribute to Don McLean | Rational Survivability @Beaker’s rockin’ tribute to Don McLean, inspired by Amazon’s EC2 woes. [If you aren’t reading Beaker, you are missing out on tremendous cloud knowledge…

Invisible Man – HonestlyWTF This is just cool. I had a hard time finding the artist in the first shot.

“Liu Bolin camouflage’s himself into any surroundings with paint. It can take him up to 10 hours for a single shot. WOW!” …

Cloud Connect Roundup Podcast : elemental cloud computing Saturday [the 12th] morning, I joined Dave Linthicum on his cloud computing podcast to discuss our impressions and findings from Cloud Connect. Learn what Kung Fu Panda is doing in th…

Is an Enterprise Data Warehouse Still Required for Business Intelligence? | TheVirtualCircle Colin White, questioning whether ‘the center holds’.

“Why is this discussion important? The main reason is at present business intelligence i…

Pinning Down Cloud Computing – WSJ.com Is Cloud Computing a backlog killer?

“But while improved cost-efficiency and greater business agility are attractive, what really excites cloud enthusiasts are the macro-economic possibilities.

Many …

Three word definition of Enterprise Architecture: Reduce Unnecessary Effort – Inside Architecture – Site Home – MSDN Blogs
More wisdom from Nick Malik: “I explained that I am an Enterprise Architect, he asked what that is. I got my chance to use my ne…

Column 2 : SAP Analytics Update
I was on this call. Nice of Sandy to document it for all of us. I’m interested in the Event Processing ties. Will get more info from SAP on Event Insight closer to the Feb 23 launch.
“A group of bloggers had an update…