A hierarchy of systems

I want the minimal maintainable systems in my life that allow me to make beautiful and/or useful things. My primary systems focus on time and project management.

Here is my timeline of system development (also rank-ordered for quality of system):

No system

A system I made up

A system collected from random parts of other systems

A complete archaic system

A complete modern system

A complete modern system modified to my personal situation

Take calendars. In the past, I did not have a calendar. I apologize to anyone who had to deal with me back then. I don’t know how I got where I wanted to be at the right time and place. I went through the hierarchy of systems, step-by-step. Right now I use Google calendar with a subcalendar for each of my different roles and responsibilities. Because it works, I spend more time using the system than working on the system.

From counts to models

Almost all data science starts with counts[1]. How many people are clicking which box? How much time are people spending on a particular page?

Only after that stage data science does gets complex (and more interesting).

There is a similar development from descriptive to inferential statistics in other sciences. Measures of central tendency (i.e., mean, median, and mode) and variance are first calculated, then models and model comparisons (e.g., regression and ANOVA) are applied.

Social media is just now entering the count stage.  Most businesses have a social media presence (binary - yes or no). They have reached a critical mass of data and are starting to organize it. The organization is counts and sums. Very few organizations are thinking about moving to the social media strategy stage (model comparisons - making choices based on data).

Many problems have followed the same pattern. Today's data scale is larger, but the analysis has the same stages. The same class of solutions can be applied at each stage.

---

1. A data scientist or statistician is usually brought in after data collection is well under way and there is an realization of its potential value. Collecting the best data in the right format probably did not happen. The classic, “I should have been at a much earlier meeting.”  ↩
   

The new gravity of San Francisco

I'm witnessing first hand San Francisco's latest tech wave.

I have been talking to many companies that are in the process of moving their offices to San Francisco proper. Other companies are apologizing for having to take shuttle buses to their palatial Silicon Valley estates.

It is interesting, possibly ironic, that Internet companies which are known for being digital are hamstrung by their physical infrastructure.

Stuart Firestein from ted.com

My thoughts:

Science is not perfect but it is the best method for improving knowledge. It makes mistakes on many levels but has built-in mechanisms to correct itself, a that process could take decades (or even centuries). Almost all other ways of improving knowing are less rigorous.

Stuart Firestein finds teaching students not exhilarating. He exemplifies one of the many fundamental flaws in R1 universities. The culture at these institutions views introduction courses (or any courses) as a necessary evil. Remember this when you are a freshman student at a big name school. These institutions are not primarily designed to give you a world-class education. It has other goals - research and grants.

It is clear from the video Stuart Firestein views himself as a “sage on stage.” Another fundamental pedagogical flaw in R1 universities (and TED).

I have taught Sensation and Perception twice. I designed the classes to show how much knowledge there is learn. I would point out where the book was wrong, incomplete, out-of-date, and suggested further research. I encouraged students to do the same. One of my goals was to inspire the desire to contribute to the body of human knowledge.

In addition to the growth of the scientific literature, there is the growth of general human knowledge. My response is not to follow but search. Even within the domains that I am “expert,” I willfully do not stay current. I spend my time solving problems. If those problems required additional “facts,” I look those up. I rather spend honing my general analytic skills than skimming RSS feeds. It does make for awkward conversions at the water cooler when someone asks me if I am familiar with a particular work or person.

Metaphors are powerful heuristics to organize thoughts. My metaphor for science is Agile Programming. I choose the next, best, completable product/problem. I ship working code/knowledge to the world in sprints. At the end of a sprint, I choose the the next, best, completable problem given how the world is and who I am at that new moment. The focus is on the doing of science for the world.

Testing serves another fundamental role in learning. Testing empirically show the limits of self knowledge. It shows where more time should be spent to continue the learning process.

PS. I think Stuart Firestein makes interesting points and does fine research. I disagree with him without being disagreeable.