The UX of a Great Software Review Site

Most software review and analyst sites miss the mark for me.  There is a fundamental reason why…I want to see the software in action!  Why can’t these sites follow this basic formula:

  1. Standard Use Cases in clear language. Reviewer comes up these.  I would even ask the software companies to come up with their own use cases.  Some will benefit one vendor, some will benefit another.
  2. Screencasts accomplishing the case. Each vendor can add whatever audio track they want, but the video has to actually show the product and each step of the use.
  3. Replacement screencasts should be allowed at any time.  Submitted to the reviewer, to make sure they are acceptable.

It’s the job of the reviewer to provide a place where people can make informed decisions and to editorialize.  This formula above would do exactly that.  What better way to choose a vendor than to actually look at how it’s used in identical business cases?

Of course, some vendors won’t want to participate.  They will say, “We don’t want to just show the product out in the open like that.”  I would absolutely include those vendors in the review and call the vendor out saying, “Vendor could not prove they could do this case”.  Any vendor is going to feel the pressure to provide the data.  Making screencasts is pretty easy.

My main point is:  Review sites are not for consumers, they are for the vendors.  I would go so far to say that a “system” could be set up for reviews of all software and other services like hosting, or javascript libraries.  The system would allow new players into the space.  It would allow editors to set up use cases and review the content to make sure it’s acceptable.

It’s not a billion dollar idea, but it’s needed in the marketplace.  If I ran the Better Business Bureau or the Consumer Protection Agency, this is the kind of thing I would build.

Whatya think?