myShoggoth

The Cycle of Scientific Discovery

There's a cycle to scientific discovery and its integration into every day life, similar to The Water Cycle.  I was going to wait to post this until I had made up a nice graphic, but I have no idea when I'd get to it.

Incubation

Those that wish to build the future must stand on the shoulders of giants.  There is a common myth that real breakthroughs will somehow come from those who are innocent of preconceptions and thus find things others aren't even aware to look for.  I don't buy it, at least not in most scientific fields at this point in history.

There will probably be exceptions that prove the rule, but even the most famous example I can think of is misrepresented.  The problems he solved were not considered unsolvable, and one of the two was being solved by someone else around the same time.  The student was a PhD candidate, hardly a tabula rasa.

The Incubation Period for our society is getting longer, and this is not a coincidence.  Our total amassed knowledge is growing at an incredible rate, and fully understanding just one vertical slice of it now takes many years.

To push the frontiers of knowledge requires that understanding, if for no other reason than to avoid duplication of effort.  Can you imagine how awful it would be to dedicate your life to studying something, make a big discovery, and find that others had already worked that out 50 years ago?  Oy.

Discovery

This is the part most people associate with science.  Once you know what's known, you figure out the unknown and try to turn a chunk of it into new known.  This is where the Scientific Method comes into play, the unknown is a giant puzzle and solving the remaining pieces is very hard, it takes time and dedication to make a hypothesis, test it, prove yourself wrong, and repeat.  Over and over.  Until you can't prove yourself wrong, and you might be a step closer to really solving that piece.

That's what it is about, not the sexy big discoveries that get lots of press, but the steady chiseling away of the unknown and refining it into Grade A Known.

You'll notice that up until this point in the cycle we're mostly talking about the education system, and in Universities this full range is carried out.  That's part of the reason there's a mini-cycle here, these new discoveries can be quickly folded back into the curriculum (normally at the graduate level).

Invention

Here is the process of turning the discovery into something useful.  Turning Unknown into Known is a wonderful thing but it doesn't put food on the table, a roof over your head, or gigabytes into your RAM.  Someone needs to take the new information, combine it with pieces of previously known and built things, and create inventions.

The primary example of this is electricity.  It had been known in static form by the ancients, and of course Benjamin Franklin famously figured out that lightning was electrical.  But it wasn't until the late 19th and early 20th centuries that useful inventions were created that took advantage of all the new knowledge.  People are still working on improving and creating new inventions based on it today.

Distribution

Even then, light and electrical power were not in everyone's hand the instant the inventions were made, nor at the discovery of electricity and its properties.  The logistics of getting the inventions into the hands and lives of everyday people and adapting those inventions to be relevant to more people is critical to the cycle.

Today we divide the planet largely into those areas which have more distribution of the results of scientific progress (first world) versus those that have less (second and third world).  Plumbing has been known for millennia, many medicines for a century or two, electricity for a century, automobiles for about the same, computers for more than half a century... but in none of those cases can you say that the distribution of inventions based on that knowledge is complete.

The speed of distribution of new inventions is increasing, at least in the parts of the world where the logistical infrastructure is there to enable it.  One of the very interesting phenomena of the past decade has been the proliferation of mobile phones in areas of the world were twisted pair phone service was spotty, poor, or non-existent.  That's been called a "technology leapfrog", where entire technology generations are skipped for newer ones that not only work better, but are easier and cheaper to deploy.

There's another mini-cycle here between Invention and Distribution.  As early inventions are created and put in front of the public, the feedback from trying to sell them cycles back into the invention process, where changes and improvements are made for another try at distribution.

Integration

The last part of the cycle is the integration of the new knowledge and inventions into the every day lives of the people who use them.  They start as new and novel with early adopters, and eventually become a part of the fabric of the culture.  I'll use mobile phones as the example for this one as well, since it is pretty obvious.  In the 1980s, few people had them and they were as big as bricks.  Many people couldn't think of a good reason anyone would ever want one.

Now it is odd to find someone who doesn't own one, and it is becoming more and more usual to find people who no longer have landlines.  They've gone from a brand new technology to part of society, and now everything that comes after does so within the context of a society where mobile phones are pervasive.

More importantly to the cycle, though, the fact that such things are commonplace and understood, or at least accepted, means that future Incubation (education) is done with access to better and more pervasive tools, technologies, and inventions, which then allows a more efficient way to gain new Discoveries.

After all, if I live in a world where I can get to most places on the planet within a day, I can communicate with a large portion of the world's population in realtime, I have easy access to clean, safe drinking water and quality food... how can I not expect us to be able to do anything we set our minds to accomplish?

Conclusion

It is easy to be complacent about the state of the cycle you participate in, and to think that the mini-cycles are a good workaround for the problems we face in going full circle.  I think the most useful discoveries are the ones that go productively through the entire cycle.  They're part of our core education, part of what every scientist builds on regardless of their field, and one of the many tools available for inventors.

The biggest weakness we have today is in Distribution, logistics being the primary bottleneck in the cycle.  The education/research system has big flaws, but overall it works and has produced incredible advances.  The invention/distribution system varies more, some fields (especially those with lower barriers to entry for startup businesses, think internet) do this amazingly well, others have made very little progress relatively speaking.

Even in the best cases, logistical barriers are keeping a large percentage of the world's population from enjoying the benefits.  Do they have computers?  Electricity?  Are they subsistence farming and have far more important concerns on their mind day to day?

A focus on Distribution as a means for making the scientific discovery cycle more efficient and productive can only get more of humanity involved as producers and consumers of scientific progress, feeding back into the cycle and helping everyone.

The Upside from Bussard IEC Fusion

Here is a post on advanced nanotechnology on the kinds of things (particularly space things) that would be enabled if they can prove it.

What excites me is all the things this technology can enable, and how far along the research is.  Not only could it get enough people and materials to the Moon, Mars, and Titan to get viable colonies started and finally get our foothold in space, but it is also the right solution to power them and keep them going.

The two most basic things in the economy are power and raw materials, from them everything follows.

Another post at the same blog gives a breakdown of how much oil is used, both in total terms and percentages of the total, by use.  The point to me is not so much that we need to reduce the amount of energy used; we need to leap ahead a generation or two on how we produce it.

Not only in terms of the environmental cost, but also in cost cost.  How much easier would it be to deal with global poverty if power becomes so cheap to produce that international charities set up big generators all over the globe?  Dealing with clean water and many issues become incredibly easier.  Not all problems, of course, but freeing people and resources from the issues that are solved to tackle the remaining ones is what progress is all about.

Daily Show Fix

In the words of Kevin Smith, never a good idea to attack the dude with the microphone.

RememberTheMilk Update

A few weeks ago I talked with my wife about buying a "Pro" account at RememberTheMilk.com, to support that site (I don't use anything that's Pro-only at this point).  She readily agreed (and that's saying something) because I've been able to stay on top of things a lot better since I started using it.

For instance, I keep all our household bills in one of the tabs with their due dates and appropriate recurrences.  Twice a month I collect all the bills and mark them off in RTM and there isn't any guesswork about whether I missed one, whether I got to any too late, etc.  Regular household chores like trash night and even irregular ones that I don't want to forget about, like clearing out the blackberry vines near the peach tree, all tracked.

Anyhoo, I'm very happy with their stuff.  They just posted an update to the site which has some very cool features and nice touches, reinforcing my happiness.  It also talks about many existing features, and I realize how little of the power of the site I'm using.  Sweeeeeet.

They're on my list of people I need to take to lunch next time we're in Australia.  Maybe I should store that list at RTM.

Interwood (or Hollynet?)

I'm a fountain of weird ideas recently; here are some thoughts on business opportunities around the WGA Strike, or more specifically around the industry changes that have led to the root causes of the strike.

The Industry

We're talking about the whole entertainment industry here: TV stations, book and magazine publishers, and music labels are all in the same basic business.  They're competing for your time, attention, and dollars, with each other and other stuff in your life.

There are a small number of massive conglomerates that control an overwhelming amount of this media, especially in TV and films.  This is a natural consequence of the structure of the industry, which has been dominated more by the cost and complexity (and value) of the distribution networks than production costs.  You can make a TV show, but how do you get it on the air, into syndication, into stores on DVD?  Companies that build up those networks have been bought by one of the big conglomerates, more for the value of the networks than anything else, and those distribution networks made part of the larger one.  Redundant parts are eliminated, efficiencies created, money is made.  This consolidation has been ongoing for decades, and would naturally continue until there are only two or three left.

The Change

So what's different now?  The Internet is.  Their massive competitive advantage disappears as the barriers to entry for all of these forms of entertainment approach zero.

If I want to write, I can just start throwing stuff up on web pages for the whole world to see for free or not very much.  And build up a massive audience (hi mom!).

Musicians can record their compositions or play their instruments and put up mp3s.  They can get some inexpensive video equipment and post their performances to YouTube or similar sites.

Animators can create entertaining shorts and post them, and build up a following.

More importantly, over the past few years people have started actually figuring out how to monetize all of this.  Build up a fan base, host ads, sell merchandise, make a good income.  You may not create a multi-billion dollar Borg-o-tron big company, but people have built very successful businesses and made a good living from things that used to require publishers/syndicates/labels to actually make money with.

The Opportunity

There has been much wringing of hands about the death of the skilled writer/musician/director/actor/animator/illustrator in this new world, and I think the opposite is the real truth.  There will certainly be a lot more crap out there purely due to the lower barrier to entry, but the same thing changes the game for those who have real skill and talent.

One common thread I've seen across many of the posts on writers' blogs about the strike is that the nature of the conglomerates means smothering executive oversight of shows, destroying creativity and generally killing most of the originality and risk.  The Internet is different, the target isn't a massive hit that appeals to everyone and wins its time slot, it is entertainment that appeals strongly to a vertical silo.  You can take risks; you can do things that are very different, because there's a good chance the audience for what you're doing is out there waiting for it, and is profoundly bored of the mostly bland crap out there now.

The Business

The big opportunity I see here on the business side is a matador-like sidestep of the current business models.  Take all the things they're struggling with, the advantage of not having entrenched interests, and make a stake in the ground in the place the industry will have to go eventually, but doesn't want to.

Distribution

First, bypass existing distribution channels completely.  This is the really hard part, but it is much easier than it was a few years ago.  Ad-based web content plus web store merchandising, the item the AMPTP is claiming they want to study the viability of for the next three years?  That's the entire business.

Creation

Work with the people who create the content, not against them.

Give them the stake in the success, sign mutually beneficial union contracts in the spirit of building the enterprise for everyone's benefit.  Create a bigger pie, don't fight so much over the size of the slices.  Work with them to make sure everyone is treated fairly and well.

Give them creative control, don't play it safe.  There's an audience out there for most anything well done; give the people who make the high quality entertainment the opportunity to find their audience.

Accounting

Competing in an industry famous for its unethical account practices, I say don't just be ethical.  Be open, to a ludicrous degree.  Take a cue from the Open Source Software movement, don't just be more open than the big guys, not just more ethical.  Be completely open, be open about absolutely everything, open all the books at all times for anyone who wants to see them.

I would not blame employees of the media companies for distrusting everything they say, especially about money.  The only way to take advantage of that, the only way to really gain the trust of the creative talent and workers who actually make it all happen is to be so open that even they are uncomfortable.  And there's no way the big boys would follow you down that path.

Why The...

The point of all of this is to have the company become The Place that the most creative, most talented, most skilled people in the industry want to work.  The place that new ideas are pitched to first, whose offers are accepted instantly, who has the reputation for the best work and the top people.

The focus of the management is to constantly drive towards that goal, and while naturally there would be lots of oversight to make sure the costs are kept under control (they'd be public knowledge; with everyone being able to look at your books you're certainly motivated to make them not only honest but totally defensible) I think the profitability would come naturally from the incredible creative content that would quickly and reliably be created.

It sounds like it would be an absolute blast, wish I had the money and contacts to make a go of it.  I don't think the startup costs would be too bad, but rule #1: Pay people for their work, don't try to bootstrap it on the cheap.  I don't think you get a reputation for being sensible when you do things on the cheap, I think you get a reputation for being cheap and wanting to get away with things.  If someone does something valuable the only real way to show it is to compensate them for it accordingly.

TalentSpring Beta 2

TalentSpring Beta 2 is live now, with pretty big improvements to UI.  I haven't checked everything out yet, but I got a demo a couple weeks ago from the CEO (as an ongoing investor in the company).

Big step forward, I'm hoping for the best for everyone at the company.

I'm also really hoping they're done with big-bang releases and can get into a better rhythm.

Plato's Northwest Drivers

I know I've already complained bitterly about the quality of Northwest Drivers, but this morning on my way to work I witnessed such a Platonic Idealist example I just have to share.

Said driver decides, too late, that he wants to turn left at an intersection.  Since we're in the PNW, he naturally shifts into reverse and starts backing up so that he can switch to the turn lane, which has vehicles in it that he pulled up next to.  I'm driving up behind him at this point, and switch into the right lane.  I wish I was more upset or shocked or something, but I'm pretty used to this.

Once he's in a position to switch lanes, which for some reason involves taking up part of the right lane as well, he just stops.  At least one car has to work their way carefully around him in the right lane to get by.  I think at this point he has noticed that the city put up barriers to keep people just like him from making a left into the Safeway (grocery store), which has a nasty driveway right before the intersection.  Locals now know exactly where and what I'm talking about.

After a minute or so of just sitting there taking up multiple lanes, he goes straight (back to where he started this whole thing).  But that's not the best part.

When the left turn folk finally get their light, he cuts one of them off and makes the left turn at the intersection after all.

Writers Guild Strike

I'm not normally a union guy, but I am a 'use your brain' guy, and thus I don't think you should be trying to fit reality to your preconceive notions of it but rather let evidence drive the theories.

I believe historically unions were required to fix horrible conditions and situations, but as they are successful their utility decreases more quickly than the changes they cause.  Over time they become like most human institutions that put the institution itself above the individual, and cause more harm to the people they were created for than they help them.

That doesn't mean they have no purpose in modern society, just that they tend to outstay their welcome... again, like most institutions.  There are probably unions out there that through the effort of individuals keep it focused on the big picture and the real (rather than nominal) welfare of its members, but I suspect those are in the minority.

I think the causes for the current WGA strike are reasonable and that the union/strike are probably the only realistic ways to get those issues addressed.  The best writeup I've seen is from Kung Fu Monkey, whose blog I started to follow a few months ago for various reasons.

Since you guys are on strike, and as the joke goes you can't write new, creative picket slogans because you're on strike, I'll donate the following:

One Two Three Four Insert Witty Slogan Here

Software Development Process Consulting

I had a flash of inspiration this morning that my ideal job would likely be consulting on how to manage and improve software development production.  Not stuff like showing people how to use UML (although I've taught that professionally in the past), but nitty gritty stuff, like making sure that all the right tools are being used, things are being planned in a sane way, people are coding consistently with the quality of the product desired, etc.  From product planning and verification all the way through development, deployment, support, and updating.

• Tools Evaluation: There are a set of tools you need to produce quality software, they need to be used together and how team members work with them should be well understood.  These are not rules for rule sake, each one exists to improve the productivity of the team members.  As an example, here is what I came up with for TalentSpring.
• Planning and Scheduling: What work is important, how do you evaluate that, how much can be done in what period of time?  When are you actually ready to give the product to customers, internal or external?  Small mistakes compound here, leading to software released well before it should have been, deadlines consistently missed, or products irrelevant to their supposed target customers.  Really engaging with the developers who will be writing the code and getting them productively involved with this stage is sadly rare, and that lack leads to many problems.
• Release and Support: Once the product is finished it is far from done, even with the best teams there will always be nasty bugs that sneak through.  Fixing them and getting the updated version to customers in a painless way that doesn't cause more problems can mean the difference between viral word of mouth recommendations and failure.  Triaging the issues quickly and correctly makes sure the only changes that are made are the ones that absolutely need to be.
• World Class Software: I wrote an entire series of articles many years ago about what it takes to make world class software.  I re-read it a couple weeks ago and it stands up very well.  I not only know what needs to be done, but have written the code.
• Evaluating Technical Talent: I've worked with some of the best in the industry, I've managed people from stars to... not stars.  I can help management evaluate their team so they know what strengths and weaknesses they have to work with, and the specific types of talent they should prioritize hiring.
• Community Engagement: In both my technical teaching as well as product team experience I've seen how important it is to appropriately engage with the user community to get feedback and keep them involved and informed with the product team.  Communication is even more important now, where companies are often able to differentiate more on user communities than on product features.
• Top Talent Mentoring: The job market for top technical talent has exploded recently, and they are in tremendous demand.  If you want to retain your top people, they need to feel like they're both valued and growing.  If you do not have the resources available in-house to mentor and help them grow it is easier for the big companies to lure them away.  I can provide technical and career mentoring.
• Bridging the Technical/Non-Technical Gap: People communicate in many different ways, I have experience working with many different disciplines and can help companies develop better communication between groups so everyone is working towards the same goals and valuing each other's work.

I think there's a lot of potential here.  I know I'd enjoy it (I have in the past), the trick is of course getting it started, building a reputation, getting successes under my belt, and some word of mouth spreading to those who could use help with any of the above.

Anyone have more direct experience with this sort of thing?  I want to strictly avoid the buzzword bingo style and the get in the door and make them regret the day they met you (but you have sacks of cash) style.  High value, build a top tier reputation, walk away with people thinking they got a bargain.

I'm sure there are other great ways I could add value that I haven't thought of, but the list above looks pretty good to me.  Not all companies/teams need them, but those that do are likely being badly hindered by them.  Looks like I may be doing more networking.