Kategorien
Uncategorized

NoSQL is Out and NewSQL is In – Says Google (Google Spanner)

Google recently released a paper on Spanner, their planet enveloping tool for organizing the world’s monetizable information. Reading the Spanner paper I felt it had that chiseled in stone feel that all of Google’s best papers have. An instant classic. Jeff Dean foreshadowed Spanner’s humungousness as early as 2009.  Now Spanner seems fully online, just waiting to handle “millions of machines across hundreds of datacenters and trillions of database rows.” Wow.

The Wise have yet to weigh in on Spanner en masse. I look forward to more insightful commentary. There’s a lot to make sense of. What struck me most in the paper was a deeply buried section essentially describing Google’s motivation for shifting away from NoSQL and to NewSQL. The money quote:

We believe it is better to have application programmers deal with performance problems due to overuse of transactions as bottlenecks arise, rather than always coding around the lack of transactions.

This reads as ironic given Bigtable helped kickstart the NoSQL/eventual consistency/key-value revolution.

We see most of the criticisms leveled against NoSQL turned out to be problems for Google too. Only Google solved the problems in a typically Googlish way, through the fruitful melding of advanced theory and technology. The result: programmers get the real transactions, schemas, and query languages many crave along with the scalability and high availability they require.

The full quote:

Spanner exposes the following set of data features to applications: a data model based on schematized semi-relational tables, a query language, and general purpose transactions. The move towards supporting these features was driven by many factors. The need to support schematized semi-relational tables and synchronous replication is supported by the popularity of Megastore [5].

At least 300 applications within Google use Megastore (despite its relatively low performance) because its data model is simpler to manage than Bigtable’s, and because of its support for synchronous replication across datacenters. (Bigtable only supports eventually-consistent replication across datacenters.) Examples of well-known Google applications that use Megastore are Gmail, Picasa, Calendar, Android Market, and AppEngine.

The need to support a SQLlike query language in Spanner was also clear, given the popularity of Dremel [28] as an interactive data analysis tool. Finally, the lack of cross-row transactions in Bigtable led to frequent complaints; Percolator [32] was in part built to address this failing.

Some authors have claimed that general two-phase commit is too expensive to support, because of the performance or availability problems that it brings [9, 10, 19]. We believe it is better to have application programmers deal with performance problems due to overuse of transactions as bottlenecks arise, rather than always coding around the lack of transactions. Running two-phase commit over Paxos mitigates the availability problems.

What was the cost? It appears to be latency, but apparently not of the crippling sort, though we don’t have benchmarks. In any case, Google thought dealing with latency was an easier task than programmers hacking around the lack of transactions. I find that just fascinating. It brings to mind so many years of RDBMS vs NoSQL arguments it’s not even funny.

I wonder if Amazon could build their highly available shopping cart application, said to a be a motivator for Dynamo, on top of Spanner?

Is Spanner the Future in the Same Way Bigtable was the Future?

Will this paper spark the same revolution that the original Bigtable paper caused? Maybe not. As it is Open Source energy that drives these projects, and since few organizations need to support transactions on a global scale (yet), whereas quite a few needed to do something roughly Bigtablish, it might be awhile before we see a parallel Open Source development tract.

A complicating factor for an Open Source effort is that Spanner includes the use of GPS and Atomic clock hardware. Software only projects tend to be the most successful. Hopefully we’ll see clouds step it up and start including higher value specialized services. A cloud wide timing plane should be a base feature. But we are still stuck a little bit in the cloud as Internet model instead of the cloud as a highly specialized and productive software container.

Another complicating factor is that as Masters of Disk it’s not surprising Google built Spanner on top of a new Distributed File System called Colossus. Can you compete with Google using disk? If you go down the Spanner path and commit yourself to disk, Google already has many years lead time on you and you’ll never be quite as good. It makes more sense to skip a technological generation and move to RAM/SSD as a competitive edge. Maybe this time Open Source efforts should focus elsewhere, innovating rather than following Google?

Related Articles

So I guess this will lead the discussion in a new direction…. again 😉

Kategorien
Uncategorized

Adobe releases 500 aweseome fonts for free!

Media_httpt3ndenewswp_dmghg
via t3n.de

With the release of the new „Edge“ product line geared towards web developers and designers Adobe just released a library of 500 free fonts.

Kategorien
Uncategorized

Learn all the cool tricks about Sublime Text 2 with this great video tutorial

I really like Sublime text 2 and I love this tutorial!

https://tutsplus.com/course/improve-workflow-in-sublime-text-2/

 

Kategorien
Uncategorized

Video preview of the very promising upcoming MySpace Relaunch

[vimeo http://www.vimeo.com/50071857 w=500&h=283]via vimeo.com

If this is not a fake, MySpace might actually rise from the dead. I’m quite impressed by what I see.

Kategorien
Uncategorized

Jam – The JavaScript package manager

Jam

For front-end developers who crave maintainable assets,
Jam is a package manager for JavaScript.
Unlike other repositories, we put the browser first.

View on GitHub Find packages

Like!

Kategorien
Uncategorized

Javascript Must Read: A gentle introduction to Backbone and Ember

So you’ve accepted the challenge to go thick on the client-side; well done. You’ve considered all the frameworks out there and are unsure which one to choose? You’re not alone. Read on.

My experience, when learning the way of writing client-side apps is proving to be steep and hard. It’s not easy to deliberately choose to use MV* on the client for someone who wrote JavaScript, based entirely on jQuery and its plugins. This is an entirely new paradigm; it requires basic programming skills and a considerable understanding of JavaScript (the language) design. If your experience relates to mine, then read on!

I will be explaining the main differences between two of the most popular JavaScript clientside frameworks: Backbone.js and Ember.js. Each of these tools has strong points, as well as weaknesses that might help you make a more thoughtful choice.

Disclaimer: as software professionals, we must deal with diversity of opinion. Backbone and Ember are results of opinionated and experienced professionals, like you and me. One tool isn’t better than the other; they just serve different crowds and, ergo, solve different problems. Thanks Trek for the solid advice.

The Philosophy

Backbone is much easier to learn than Ember.

First and foremost, you need to understand that Backbone and Ember particularly serve slightly different crowds. Regarding complexity, Backbone is much easier to learn than Ember. However, it’s said that once you learn Ember, it hardly gets any more complex. Take Trek’s word on it. If you’re just getting started with some real JavaScript, then perhaps Backbone is your tool. If, however, you know that you’re going to deal with a lot more than just a simple use case or two, then you might prefer Ember.

Backbone

Jeremy Ashkenas built Backbone so it would be possible totake the truth out of the DOM. What he means by this is: whatever business you did using only jQuery / Mootools / Prototype could and should be better extracted into pure JavaScript structures – objects, if you will. Instead of using DOM elements to define your business elements and behavior, Backbone invites you to do it the other way around. JavaScript objects are the core and the DOM is merely a representation of that data.

With Backbone, you have some given assertions:

  1. Data lies in JavaScript objects, not the DOM
  2. Event handling lies in JavaScript objects, not jQuery event bindings
  3. The way you save data in a backend server is done through the objects that contain the data

You are given complete control over the way you build your app. Backbone was meant to give you a basic way of designing your model objects and how these interact with each other through event bindings.

Rendering HTML to the DOM is of your responsibility. You are free to choose any template engine: Mustache, DoT, Handlebars, Underscore, etc. Backbone contains a View prototype that has the responsibility of articulating the DOM and your JavaScript core.

Ember

When Tilde started building Ember, it did so with a far more challenging goal: to provide standard conventions in client-side development, eliminating as much boilerplate as possible. The result is a much more ambitious framework that aims for a predictable architecture and steady development.

Ember shares some common points with Backbone in the way it tries to pull data and behavior out of the DOM by providing extendable JavaScript prototypes, but it does this in a very different manner than Backbone does.

Ember stands on:

  1. Two-way data binding: objects in Ember are able to register bindings between one another. That way, whenever a bound property changes, the other one is updated automatically.
  2. Computed properties: if you wish to have a property that is a result of a function, you can create them and assign a property as computed by that function.
  3. Template auto-updates: when an object is updated in your app, all the views currently displayed in the screen that are bound to that object automatically reflect the change, with no boilerplate.

The DOM – Views

Both Backbone and Ember have common key concepts, such as views. They both represent DOM communication, respectively. The way they accomplish this concept are somewhat different, though.

I’ll use the Todo use case for the examples below, inspired by the TodoMVC showcase.

Backbone

A Backbone View might something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
var TaskView = Backbone.View.extend({
  tagName : "li"
, template : "task-template"
, render : function() {
    // your code to render here.
  }
, events : {
    "click .mark-done" : "mark_as_done"
  , "change .body"     : "update_body"
  }
, mark_as_done : function() { /* code here */ }
, update_body  : function() { /* code here */ }
});

This is simply the definition of your view. You will need to instantiate one if you want it to be in the page. Something like this will do the trick:

1
2
var task_view = new Task({ model : task_model });
$("body").append(task_view.el);

Notice that we’re passing a model in so you can keep a reference to the data object that feeds the template. The template property inside the view can be used to call an outside template, via an identifier. I’ve used something like this in the past:

1
2
3
4
5
6
7
8
9
var TaskView = Backbone.View.extend({
  template : "#task-template"
, render : function() {
    this.$el.html(
      Mustache.render($(this.template).html())
    , this.model);
  }
  // snip
});

Ember

Ember has a different approach to views. In fact, the convention states that views should talk to controllers and not models directly. This is a good practice, if you intend to follow a stable architecture. I’ll explain the sample for the same view:

1
2
3
4
5
var TaskView = Ember.View.extend({
  templateName : "task-template"
, mark_as_done : function() { /* code here */ }
, update_body  : function() { /* code here */ }
});

That’s it. But where’s all the rendering stuff? Well, Ember lifts that boilerplate for you. Simply say what the template is, the controller that holds the data object, and then you just need to append it to the DOM.

1
2
3
4
var task_view = TaskView.create({
  controller : task_controller // Ember.ObjectController
});
task_view.append();

When creating a new view instance, it will bind the controller’s content (which can be an Ember.Object or a list of them) to the view. When you decide to append the view to the DOM, it will look up the template and place the generated markup for you.

Thoughts

Backbone is more explicit and less magical.

Backbone is more explicit and less magical. You create a View, tell it what template to use and how, register the events and do what you have to do. They own the page. That’s a great start for those coming from a jQuery background. However, when something needs to be updated in the DOM, you will face some boilerplate.

With Ember, updates are automatic. You say what template it is and event callbacks are functions inside the view object. Any time an object is updated, the view automatically updates the page.

Some common event bindings are built into Ember and others must be put into the template. It’s good for those who come from a backend perspective, as it reduces boilerplate in a considerable way.

The Data – Models

Models in Backbone and Ember are quite similar. They hold information for a business entity.

Backbone

An example of a Backbone model looks like this:

1
var TaskModel = Backbone.Model.extend();

With this simple line of code, you have a working model with RESTful communication built-in. You get methods like save to persist the data and fetch to load it for free; no plugin is required. Validation is also built into the way data is saved by providing a validate callback, which returns a boolean that tells the record to be saved or not. The implementation of the validation is still for the developer to do.

To create a new task, you instantiate a new TaskModel.

1
2
3
4
var task = new TaskModel({
  body : "Mow the lawn"
, done : false
});

You may inject as many attributes as you like, because the task’s attribute list isn’t strict (think of it as schemaless). You can still set a defaults property when extending Backbone.Model.

Ember

With Ember, there are no models, just objects. It might look something like this:

1
var TaskObject = Ember.Object.extend();

Similar to Backbone, you need to extend from Ember.Object to create an object class. It inherits all the basic functionality for a class with callbacks for when it gets changed, created and destroyed, amongst other features. It does not, however, have backend communication out of the box. Ember.Data is being developed as an extension of Ember.Object by the Ember core team to fulfill that need. It’s already usable but not stable as far as the documentation tells.

Ember objects are also considered to be schemaless. To inject defaults into Ember objects, you extend Ember.Object by passing an object with as many attributes as you require.

1
2
3
4
var TaskObject = Ember.Object.extend({
  body : "Mow the lawn"
, done : false
});

Thoughts

Backbone has a consolidated way of syncing up with a persistence layer over REST and that’s a good convention there. It’s one less thing you have to configure in order to work with a backend server.

Ember is working its way toward making Ember.Data ready for production use, and it looks promising. Even so, the particularity of Ember objects having two way bindings makes it dead easy to perform connections between objects.

At this point in your reading, you have an inflection point between Backbone’s stability in communicating with the backend server and Ember’s bindings. Whatever’s most important to you should determine your decision.

The Glue – Controllers

This is where the frameworks part ways. They have a huge conceptual gap on how to glue things together in your app. While Backbone strives to remain as simple and flexible as possible, Ember sacrifices codebase size for a better architecture. It’s a tradeoff, really.

Warning: the following examples don’t contain HTML template samples.

Backbone

As I noted, Backbone aims for simplicity that converts to flexibility and it achieves such attributes precisely through the lack of a controller class. Most of the workhorse is distributed around views, collections, models and the router (should you choose to use Backbone’s Router).

Considering a list of tasks that needs to be managed, it would require:

  • A Collection to store the tasks.
  • A Model to store a task’s information.
  • A View to represent the collection.
  • Another View to represent each task.
  • A Router to manage URLs.

Most of the application logic will live in the views, as they connect models to the DOM. There is no clear distinction of responsibilities, as the view does everything. It can be good for small applications that don’t require a solid architecture.

To display a list of tasks, you would end up with something like this:

Collection

1
2
3
var TaskList = Backbone.Collection.extend({
  model : Task
});

Model

1
var TaskModel = Backbone.Model.extend();

Views

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
var TaskListView = Backbone.View.extend({
  render: function() {
    this.$el.empty();
    for (_i = 0, _i < this.collection.length; _i++) {
      var task = this.collection.models[_i];
      this.$el.append(this.renderItem(task));
    }
    var tasks = this.$el.html();
    this.$el.html(Mustache.to_html(template, {
      tasks: tasks,
      no_tasks: !this.collection.length
    }));
  },
  renderItem: function(task) {
    var view = new Row({ model: task });
    var el   = view.render();
    return el.el;
  },
});
1
2
3
4
5
6
7
var TaskView = Backbone.View.extend({
  tagName: "tr",
  render: function() {
    this.$el.html(M.to_html(template, this.model.attributes));
    return this;
  }
});

Router

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
var Router = Backbone.Router.extend({
  initialize: function() {
    this.tasks = new TaskList;
    this.view = new TaskListView({
      collection: this.tasks
    });
  },
  routes: {
    "": "tasks_list",
  },
  tasks_list: function() {
    this.view.render();
    $(".bucket:first").html(this.view.el);
  },
  start: function() {
    Backbone.history.start({
      pushState: true,
      root: "/tickets/"
    });
  }
});

Notice that the collection doesn’t have a template of its own; rather, it delegates to a single task view being rendered and appended to the final result being put on the page.

Ember

The number of classes required to have the same setup is slightly bigger.

  • Instead of a Collection, you would have an ArrayController, which works very much alike.
  • You would have an extra ObjectController for managing a single task.
  • Instead of a Model, you would have an Object / DS.Model, which work alike.
  • You would have the same kind of Views.
  • A Router is also responsible for managing URLs.

You might be thinking that the two frameworks are not too different from one another. It’s rather tempting, but it’s not exactly true. Some particular differences are:

  1. The controller is responsible for interacting with the data objects, not the View.
  2. The views are responsible for handling the DOM, not the controller.
  3. The views communicate with the controller, not directly to the data objects.
  4. The data that feeds the view template is actually a binding to the controller’s data.
  5. The router is more of a state manager, which includes much more than handling URLs.

The separation of concerns is good in the long term. Controller handles data, views handle the DOM, period. This kind of decoupled and cohesive, boilerplateless design allows for more focused testability.

The implementation to display the same list of tasks would be something like the following, considering a full Ember application:

Application root architecture

1
2
3
4
5
window.App = Ember.Application.create();
App.ApplicationController = Ember.ObjectController.extend();
App.ApplicationView       = Ember.View.extend({
  templateName: "application"
});

Object

1
App.Task = Ember.Object.extend();

Controllers

1
2
3
App.TasksController = Ember.ArrayController.extend({
  content: []
});

View

1
2
3
App.TasksView = Ember.View.extend({
  templateName: "my-list"
});

Router

1
2
3
4
5
6
7
8
9
App.Router = Ember.Router.extend({
  root : Ember.Route.extend({
    index: Em.Route.extend({
    route: '/',
    connectOutlets: function(router){
      router.get('applicationController').connectOutlet('tasks');
    }
  })
});

In Ember’s case, there’s not much being said about how things are done inside. All of that boilerplate is taken away so you can focus on what really matters in your app: you define a task object, a task list controller with an array called content, your view and the router simply combines them all together and puts it in the page.

Thoughts

After realizing how Ember really works, it starts to become liberating.

Predictably, this segment was the hardest to grasp on both frameworks. Backbone was definitely easier to learn and its flexible nature gives control over the way objects and DOM interact. This might be good for you, if you really need that kind of flexibility but still want to maintain a structure for your app’s logic in the JavaScript side.

As for Ember, its breathtaking implementation might be scary at first. However, after realizing how Ember really works, it starts to become liberating. All the conventions the framework sets for you releases you from boilerplate and configuration, letting you focus on your app. This is similar to what Rails did for serverside development that caught so much attention.

What Sets Them Apart?

Ember was meant to lift the common burdens of JavaScript development in the browser.

So far, the whole point of showing the two tools off has been to acknowledge their single and noble purpose: to delegate power to the client-side, through both structure and method.

Backbone core strength is definitely its KISS approach. It provides you with the minimum to let go of the DOM as the core supporter of your app, and start using real JavaScript objects that can be tested and designed properly.

Backbone comes packed with collections, models, views and the router, amongst other small utilities. You are free to do what you please with them.

Ember, on the other hand, was built with a different mindset, as it aims for a much more conventional and opinionated way of building web apps. It tackles a set of common problems, such as boilerplate, data binding and DOM templating so you don’t have to worry about them from the start. Ember was meant to lift the common burdens of JavaScript development in the browser.

Ember comes packed with objects, controllers, auto-updating views, state machines, bindings, observers and a router (which is also a state machine), all of them conjured with a good dose of conventions. You have an architecture already designed and ready to begin working without losing focus.

Conclusion

Mind the learning gap. Your experience and cultural heritage will strongly dictate how fast you join the client-side. If you’re scared of what to do or which one to pick, then I struck a nerve of yours and that’s good! Want a good answer on which to choose? Both.

It’s all about the JavaScript

If you’re unsure how even jQuery does all its magic, then start learning Backbone. It’s easier to begin with, and the documentation is dead simple to read and understand. After you’re done, start building something. Go dirty. Check these tutorials if you need some help.

If you’re still in the dark, read Yehuda Katz’s entries on how JavaScript works.

Once you get a better vision of how the JavaScript works as a language, you will begin to gain a better grasp of how the objects interact with each other. When you do, go for Ember. It’s more complicated at first, but don’t give up. Start reading the docs and the guides. You might want to check Trek Glowacki’s blog entry just before getting your hands dirty.

My bottom line

Personally, I’m leaning towards Ember; I enjoy its robustness at a macro scale, and I also prefer its conventions. Backbone is a more malleable and easier tool for smaller apps or small features inside an existing app.

I’m still learning both, and have a few challenges to tackle:

  • Automatic tests: how to do them and which testing suite is better. Qunit or Jasmine? Headless (thinking PhantomJS), Node or browser test runner? Not sure yet.
  • File uploads
  • Internationalization

What are your thoughts on this whole debacle? Do you have any challenges in mind? Any difficulties or impediments? Let me know!

A nice introduction to the two JavaScript Frameworks Backbone and Ember

Kategorien
Uncategorized

Guidelines For Mobile Web Development | Smashing Magazine

Media_httpmediasmashi_dhpbq

A nice collection of articles on mobile web development

Kategorien
Uncategorized

Some nice facts about website performance

Media_httpwwwderschep_uuhyd

Sums it up nicely 😉

Kategorien
Uncategorized

The best Mobile Frameworks Comparison Chart

Mobile Frameworks Comparison Chart

Find the right framework for mobile development using this awesome comparison chart of mobile frameworks

Kategorien
Uncategorized

Usable yet Useless: Why Every Business Needs Product Discovery

Usable yet Useless: Why Every Business Needs Product Discovery

Brasília is a remarkable, bizarre city. The vision of architect Oscar Niemeyer, it was built in just four years, from 1956 to 1960. More than 50 years later, its beauty and elegance are renowned.

But Brazil’s capital city is known for something else as well: how difficult it is to live there.

A “shiny citadel” from far away, as The Guardian once wrote, up close Brasília has “degraded into a violent, crime-ridden sprawl of cacophonous traffic jams. The real Brazil has spilled into its utopian vision.”

This problem echoes across today’s web landscape as well, where the needs of ordinary users spill constantly into designers’ utopian vision. All around us we see beautiful, empty monuments erected not for their users, but for the people who built them—and the VCs who are scouting them. Even sites and apps that go beyond beauty to usability often fail because they can’t find a big enough market.

Why can’t some interactive products find enough users to be sustainable? Why are there so many failed startups, despite a renewed focus on design?

Most importantly, what can we do about it?

The rise of usable, useless products

We’ve long accepted that for a product to be useful, it needs to have acceptable levels of both utility (“whether it provides the features you need”) and usability (“how easy & pleasant these features are to use”). Yet far too often, we seem to ignore the former in favor of the latter, ending up with lots of easy and pleasant applications that have no reason to exist. One could argue that the first version of Color fell into this trap. And when’s the last time we heard something about Path?

One of the major problems that new products in particular run into is a lack of product/market fit, as Marc Andreessen has noted:

The quality of a startup’s product can be defined as how impressive the product is to one customer or user who actually uses it: How easy is the product to use? How feature rich is it? How fast is it? How extensible is it? How polished is it? How many (or rather, how few) bugs does it have? The size of a startup’s market is the the number, and growth rate, of those customers or users for that product…

The only thing that matters is getting to product/market fit. Product/market fit means being in a good market with a product that can satisfy that market.

The problem arises when startups and companies don’t spend enough time to increase the likelihood of good product/market fit before they start design and development. The Lean Startup concept of “Minimum Viable Product” is certainly useful, but shouldn’t we rather focus on Minimum Desirable Products? What’s the use of fast iteration if all it does is get us to the local maximum more quickly?

But before we get ahead of ourselves and discuss how to fix this, let’s jump into some of the all-important “why” questions.

Why products fail to fit

Brasília’s biggest problem is that the architects who designed it didn’t consider how the city would be used once millions of people were living there. They exhibited Architectural Myopia—designing for industry, not people. I’ve written before about a similar phenomenon in our industry, Designer Myopia. Lured by the recognition (and clients and VCs) they deserve, designers are drawn to being featured in galleries and list-driven blog posts that drive tons of traffic.

There is nothing inherently wrong with that need for recognition—but it is a problem when it hurts users. If Brasília teaches us anything, it’s that becoming blind to the needs of users leads us down a dangerous path where we lose control over our products, with no way to get it back. Once something has shipped, you can either iterate or pivot. Iteration is great if you’re on the right path. Pivoting is dangerous because changing course can wreak havoc on employees and users alike.

Product discovery: a better way

If we want to design better, more useful products, we need to stop designing solutions too early and start instead with product discovery: a process that helps us understand the problem properly so we don’t just design things better, but design better things.

Product discovery consists of three steps:

  • Step 1. Frame the problem and maximize the opportunity
  • Step 2. Explore and assess multiple solutions
  • Step 3. Prioritize and plan

1. Frame the problem and maximize the opportunity

It’s hard to argue with Einstein:

If I had an hour to solve a problem I’d spend 55 minutes thinking about the problem and 5 minutes thinking about solutions.

Step 1 of product discovery is that proverbial 55 minutes. Here, you should discuss and answer questions such as:

  • For whom are we solving this problem?
  • Which user needs are we trying to address? For existing products, what are the shortcomings we need to fix?
  • What customer insights do we have available to inform the solution (customer support, analytics, market research, user research, competitive analysis, etc.)?
  • How will solving this problem help our business?
  • Why is our business capable of making this solution a success?
  • How will our success be measured?

There are several techniques to structure the discussion and make it easier to get to the bottom of these questions. Fishbone Diagrams and The Five Whys are two root-cause analysis techniques that can be applied very effectively to defining a problem in terms of user needs and business goals.

This phase always—without fail—produces insights the team finds incredibly valuable. Startups gain clarity about what to say “yes” and “no” to in their product, and large corporations learn how to go beyond customer-centricity buzzwords and discover which benefits they should be selling to their users. As just one of many examples, I was once in a workshop that revealed the executives had a completely different vision for the company than the designers and developers. It was an awkward two hours, but in the end they agreed on the tough but correct decision to suspend their e-commerce plans until some of the content areas on the site had been sorted out. It’s great to see a statement of purpose emerge from these sessions—one that finally gets an organization to agree on what the product’s focus should be.

From this step, I produce a problem-frame diagram, which is simply a visual summary of the main takeaways in the form of three overlapping circles:

  • User needs
  • Business goals
  • Core competencies

Every decision the team makes should be anchored in at least one of these circles—preferably in the overlap of all three. Design decisions should focus on meeting those needs and capitalizing on the business opportunities by using the core competencies identified.

Customer journey maps are another useful output, and Megan Grocki and Jamie Thomson’s take on them is very informative. Journey maps are visual representations that help to summarize research, highlight and prioritize opportunities, and get buy-in from stakeholders.[1]

Once the problem has been defined (and agreed on by all stakeholders), it’s time to start thinking about solutions.

2. Explore and assess multiple solutions

The takeaways from problem-framing lead into a period of divergent thinking, where you produce many different possible solutions as quickly as possible—visually. Break out the pencils, and lots and lots of paper.

Rather than open your laptop too early in the design process, use sketching to produce a variety of solutions in a short amount of time. Sometimes “Move to Trash” just doesn’t cut it when you need to let go of an idea you wish you never had. There’s nothing as satisfying as crushing a bad idea and throwing it over your shoulder in disgust.

In this phase of the process, you work together to come up with storyboards, sketches, and low-fidelity prototypes to visualize ideas. It’s also a great time to start getting feedback from potential customers. And yes, let’s say it together: Everyone can draw. If Dan Roam says so, who are we to disagree?

3. Prioritize and plan

I talk to many teams who complain about “analysis paralysis”—an inability to make decisions because there are just too many factors (and people) involved. Good prioritization methods give teams comfort that even though they’re not focusing on everything at once, they are focused on the right information to make good decisions.

You can do this with a phase of convergent thinking that narrows down which ideas and solutions to explore further. There are many established processes for this type of prioritization, each designed for a different scenario:

  1. With the KJ-Method, you group similar issues together and use a voting mechanism to rank those issues in order of importance. It’s best when you have a large group of stakeholders who all have strong opinions about the product and you want to make decisions quickly.
  2. The Kano Model uses a two-dimensional axis to group issues into one of three categories: basic expectations (features that users expect as a given), excitement generators (delightful, unexpected features), and performance payoffs (features that need continuous improvement to increase user satisfaction). This method works when you want to ensure you have a balanced roadmap that addresses basic requirements, as well as innovative features that might help the product pull ahead of competitors.
  3. Amazon’s approach prioritizes large themes first, before going into individual features/projects to address those themes. It’s a good approach when the sheer number of features or improvements required feels overwhelming, and you need a way to structure and make sense of all of them.

These methods work because they facilitate teamwork without falling into the traps of “design by committee.” Everyone gets a voice, but not everyone gets to make decisions. That’s an essential attribute of any good prioritization method, because as Seth Godin says, “Nothing is what happens when everyone has to agree.”

In addition to providing the necessary structure to reach prioritization decisions quickly, these methods also produce tangible artifacts that can help you sell your ideas to internal stakeholders. User experience is often much less a design problem than it is an organizational problem. As much as we just want to do our work without obstruction, we can only be truly effective if we also make a compelling argument to people in other parts of the organization. These structured prioritization methods make that step reasonably painless by helping you produce written and visual records of your thought process.

Once done, you should be able to narrow down ideas to a select few you want to build and test—and be comfortable that those ideas have the best chances of meeting your user needs and business goals.

The output

The artifacts produced during product discovery depend on the scope and nature of the project. Sometimes it’s a few sketches on the back of a napkin that a developer uses to start prototyping; sometimes it’s a big PowerPoint document summarizing the process and key takeaways in an effort to bring senior executives along for the ride.

Regardless of the physical output, at the end of the process you should be able to answer the following questions with ease:

  • What is the problem we are trying to solve?
  • For whom are we solving it? Why should they care?
  • What’s the vision for the solution?
  • What’s in it for us?
  • What’s our implementation plan?

The real power of this process is that it will give your team comfort that you’ve introduced enough variation into the design process to ensure you’re not climbing the wrong mountain to a local maximum.

That’s fine for you

“This is nice,” I hear you say, “but we’re a fast-moving startup and we don’t have time to sit around and talk.”

You do if the alternative is failure, brought on by an unhealthy addiction to pretty things that lead to 15 minutes of fame, but not much else.

We’re entering an interesting era in web design. Retina displays might not have mass adoption yet, but it’s only a matter of time before they become the norm. We’re also seeing a level of interest in typography and graphics last experienced when color CRT monitors became a thing. There are many shiny objects out there, and if we focus on those (or focus on impressing the VCs that are focused on them) to the neglect of usefulness, we might find ourselves in a situation similar to that of only a few years ago, when we built Flash intros on every site just because we could.

In other words, product discovery is essential for startups precisely because we’re in a time of such exciting visual innovation.

We cannot let the allure of the visual tear us too far away from the usefulness of the products we develop. It is true that failure teaches us a great deal about what works and what doesn’t. But it’s so much cheaper and more effective to fail at a variety of ideas on paper than it is to fail at one full-blown, VC-backed idea. As Color can probably attest, it’s hard to come back from that.

Together, we can avoid building digital Brasílias—projects that generate buzz, but don’t meet the needs of the people who live there. So let’s discover before we build.

References

[1] If you’re new to journey mapping, see: The Anatomy of an Experience Map, Using Customer Journey Maps to Improve Customer Experience, and Building a Vision from a Journey Map.

A nice sum up of methods to improve your product