Evaluation of Solver Proposals is the most exciting, important and time consuming part of working with InnoCentive.  While you’re trying to determine the value of the Proposal based on its technical or business merit, you’re also trying to determine its novelty, strategic value, intellectual property standing and cost of integration.  To help with the evaluation, we’ve provided some helpful tips that have been time-tested by some of our most effective Seeker Clients.

Evaluation Tips

1. Try to limit the number of people who review the submissions to protect yourself from IP contamination. Usually, the owner can make a first pass and filter out a number of the submissions that are not relevant or viable. Then a larger group can spend more time on the top tier Challenges
2. Make a checklist of requirements so you can see which ones are met or addressed by the submissions. This will help differentiate submissions quickly.
3. TAKE NOTES: good notes as you filter solutions will be very useful when you go to write evaluations later on. Otherwise you will end up having to re-read all the submissions because you cannot remember why you rejected it in the first pass. This will save lots of time.
4. Usually, you can group them by approach as several will use the same type of approach even though they are very different. You can then pick the best one using that approach and put the others aside. Then you can compare the best of each approach.
5. You may want to make a matrix or a spreadsheet listing the submissions, approach, requirements met, notes etc. This allows for direct comparison on one sheet.
6. Talk with your InnoCentive IPM (Innovation Program Manager) before you make your final decisions to discuss. (Note: the Solvers will not see anything you do until the IPM approves your evaluations.

Evaluation Differences by Product Type

Ideation Challenges:

1. You retain the rights to ALL solutions but must distribute the entire award amount.
2. You just need to pick the winners and do not have to evaluate each non-winner. You should rate them with a number, but do not have to give reasons for rejection.
3. You receive the contact information of all of the Solvers that are awarded.

Theoretical Licensing Challenges:

1. The submission shall be evaluated on a theoretical basis only. You cannot conduct or require any reduction to practice or experimental validation of the proposal before a monetary award has been paid for the right to do so.
2. You can choose whether or not to purchase any of the proposed solutions for the award amount.
3. You will be granted a non-exclusive license to use any of the technology purchased.
4. You must provide reasons for rejection as described above.
5. You receive the contact information of all of the Solvers that are awarded.

Theoretical IP Challenges:

1. The submission shall be evaluated on a theoretical basis only. You cannot conduct or require any reduction to practice or experimental validation of the proposal before a monetary award has been paid for the right to do so.
2. You can choose whether or not to purchase any of the proposed solutions for the award amount.
3. You will be granted Intellectual Property Rights for any of the technology purchased as submitted in the Proposal.
4. You must provide a number rating and reasons for rejection as described above.
5. You receive the contact information of all of the Solvers that are awarded.

RTP Challenges:

1. The submission can be experimentally validated within the evaluation period.
2. You can choose whether or not to purchase any of the proposed solutions for the award amount.
3. You will be granted an exclusive license to use any of the technology purchased.
4. You must provide a number rating and reasons for rejection as described above.
5. You receive the contact information of all of the Solvers that are awarded.

eRFP Challenges:

1. InnoCentive will provide you with all of the proposals and you may contact the individuals/companies that submitted them.
2. You should discuss with the IPM whether you think you will or will not contact each Solver.
3. Contact information will be included in each submission.

We hope that the tips and guidelines above will help you manage your process of evaluating the Solver proposals that you receive.  As always, if you have any questions or need further assistance during the evaluation stage, please contact the Innovation Program Manager that worked with you during the Challenge development process.  S/he will be able to provide further guidance so that you select the proposal that will add the most value to your organization.

Hello Readers!  I am writing to tell you about some exciting news from the Client Services group at InnoCentive.

Just last week we launched an exciting new Challenge entitled Predictive Data Analysis. This $100,000 Challenge asks Solvers from all backgrounds to build a predictive model based on a complex dataset.  I know, $100,000 on its own makes this Challenge quite special. But, there is another really cool feature that’s so noteworthy – The Prodigy. This website feature allows enables Solvers to get instant feedback on how well they’re doing in comparison to other Solvers.  Sound exciting? Let me tell you a bit more.

First, a bit of background. The Predictive Data Analysis Challenge asks Solvers to build a model using a large dataset in order to estimate the relative performance of various breeds of an organism.  We have provided the molecular and performance data on 100 breeds of the organism and ask Solvers to estimate the relative performance of an independent set of 150 breeds based on their molecular data.  The Prodigy allows Solvers to upload the relative ordering of the breeds and then it will instantly be compared to the known answer provided to InnoCentive by the Seeker.  After submitting, the Prodigy will provide Solvers with their score, their ranking and if they are within the top 10 best submitted scores thus far, their username will appear on the leader’s table.

This feature is obviously not amenable to all of InnoCentive’s Challenges. However, we think that it will encourage you to continue if you’re on the right track and to go back to the drawing board if you aren’t!  We recommend that you log into the Predictive Data Analysis Challenge and let us know what you think about the Prodigy. We’d love to hear from you in the comments section below.

Happy Solving and good luck.

Gabriel Eichler
InnoCentive Client Services

We were blown away by the quality of all three video finalists – each Solver did an admirable job at conveying the power of InnoCentive Solvers to truly Change the World.   The second and third place winners were announced last week and can be seen here.  Thanks again to all who participated.

Below, you will find Bruce’s “Complexity” Digest #2, we hope you enjoy.   Please let us know your feedback and feel free to respond to the blog posts and share your thoughts and reactions with others.

The Hidden Fragility of Complex Systems: Consequences of Change, Changing Consequences, SFI Working Papers

Abstract: Short-term survival and an exuberant plunge into building our future are generating a new kind of unintended consequence – “hidden fragility. This is a direct effect of the sophistication and structural complexity of the socio-technical systems humans create. It is inevitable. And so the challenge is, How much can we understand and predict about these systems and about the social dynamics that lead to their construction?

• Source: The Hidden Fragility of Complex Systems: Consequences of Change, Changing Consequences, James Crutchfield, DOI: SFI-WP 09-12-045, SFI Working Papers

Systemic Risks in Society and Economics, SFI Working Papers

Abstract: This contribution presents a summary of sources and drivers of systemic risks in socio-economic systems and related governance issues. The analysis is based on the theory of complex systems and illustrated by numerous examples, including financial market instability. Typical misunderstandings regarding the behavior and functioning of socio-economic systems will be addressed, and some current threats for the stability of social and economic systems are pointed out.

• Source: Systemic Risks in Society and Economics, Dirk Helbing, DOI: SFI-WP 09-12-044, SFI Working Papers

Why We Conform, PLoS Biol

Excerpt:

In his book Why We Cooperate, Michael Tomasello explores the socio-cognitive mindset that forms the basis of human sociality, including the creation of cultural artifacts and social institutions. The key message is that humans are fundamentally helpful and cooperative, as evidenced by infants’ willingness to provide information, help, and share worldly goods. Later in life, experience may corrupt this benevolent attitude, but the core point for Tomasello is that children exhibit other-regarding preferences, and it is precisely this feature that sets them apart from our closest living relatives, the great apes.

• Source: Why We Conform, Julia Fischer, DOI: 10.1371/journal.pbio.1000277, PLoS Biol 8(2): e1000277, 2010/02/02

Absolute Humidity and the Seasonal Onset of Influenza in the Continental United States, PLoS Biol

Summary: Here, the authors demonstrate that variations of absolute humidity explain both the onset of wintertime influenza transmission and the overarching seasonality of this pathogen in temperate regions.

• Source: Absolute Humidity and the Seasonal Onset of Influenza in the Continental United States, Jeffrey Shaman, Virginia E. Pitzer, Cécile Viboud, Bryan T. Grenfell, Marc Lipsitch, DOI: 10.1371/journal.pbio.1000316, PLoS Biol 8(2): e1000316, 2010/02/23

El Farol Revisited, SFI Working Papers

Abstract: Some years ago Brian (Arthur 1994) published a seminal article on the problem of resolving the crowding conditions at his favorite local bar El Farol, in Santa Fe. The informal setting and its seating problems provided a striking metaphor for a basic coordination problem that occurs in many contexts. Arthur provided an imaginative and deep solution to an every day minor problem. A simple version is as follows: Say 100 people like to go to listen to the music, but all dislike overcrowding. They all have the same taste that indicates that they enjoy attending if there are 60 or fewer individuals, but would prefer to stay away if there is a higher number than 60 in attendance. Each individual has a large set of rules of thumb that he or she utilizes. The rules are of the variety such as do not go the bar if last time there were over 60 present; or go if the you think the trend 81, 71, 62 will continue. As long as an individual’s rule of thumb works he stays with it, when it fails another rule is tried. Arthur’s simulations showed that the mean attendance was around 60 although the numbers were in constant fluctuation.

• Source: El Farol Revisited, Martin Shubek, DOI: SFI-WP 09-12-043, SFI Working Papers

Identifying Prototypical Components in Behaviour Using Clustering Algorithms, PLoS ONE

Excerpt: Quantitative analysis of animal behaviour is a requirement to understand the task solving strategies of animals and the underlying control mechanisms. The identification of repeatedly occurring behavioural components is thereby a key element of a structured quantitative description. However, the complexity of most behaviours makes the identification of such behavioural components a challenging problem. We propose an automatic and objective approach for determining and evaluating prototypical behavioural components.

• Source: Identifying Prototypical Components in Behaviour Using Clustering Algorithms, Braun E, Geurten B, Egelhaaf M, DOI: 10.1371/journal.pone.0009361, PLoS ONE 5(2): e9361, February 2010

Numbers Rule Your World: The Hidden Influence of Probability on Everything You Do, McGraw-Hill

Summary:

This fascinating book from renowned statistician and blogger Kaiser Fung takes you inside the hidden world of facts and figures that affect you every day, in every way. These are the statistics that rule your life, your job, your commute, your vacation, your food, your health, your money, and your success. This is how engineers calculate your quality of living, how corporations determine your needs, and how politicians estimate your opinions. These are the numbers you never think about-even though they play a crucial role in every single aspect of your life.

• Source: Numbers Rule Your World: The Hidden Influence of Probability on Everything You Do, Kaiser Fung, McGraw-Hill, 2010/02/01

To access your “My InnoCentive” page, click the tab on the far right.  In addition, the home page will provide more information about InnoCentive, other Solvers and some of our Innovation partners.  Below is a snapshot of how the page will look:

We hope you like the new look and functionality. This is just the first step in series of website enhancements, and we will keep you posted along the way.

Another exciting advancement this week is the introduction of Solver and Seeker Discussion Forums.  You will have the opportunity to further participate in the InnoCentive community and discuss topics centered around innovation and problem solving.  Only registered InnoCentive users may post comments but all web-site visitors will have access to read the latest topics in these forums.

]]> http://blog.innocentive.com/2010/02/25/website-surprise/feed/ 2