A couple of old model fitting tricks

I picked up many modeling tricks in graduate school working in a computational neuroscience laboratory. We doing machine learning, but we don't know at the time. We called it "automated model fitting." We used custom (and very finicky) algorithms in MATLAB. Now people are blessed with scikit-learn. These ideas might help out for edge cases:

• Have good "fake" data. "Fake" data allows for testing of the algorithms. There is an art to dummy data, it should be noisy but allow the models to converge.
• Linearly transform the data so all dimensions are within the same order of magnitude. Some models have trouble with weighting noise parameters on different scales.

My recent failures

I fail all the time. I recently failed at completing several MOOCs. I'm one of the unwashed masses that started but didn't finish.

Even through attempting, I learned something from each course. Given my personal value in life-long learning, improvement is my measure of success. Completion is easier to quantify and a more commonly accepted measure of success. A "factory" model of education values completion. A half completed commodity has little value. A factory stops work on a product and ships it. My professional life is continuous delivery. Any incremental improvement adds value to a continuous delivery system. I picked up new viewpoints on existing concepts that improved my understanding of the world through "failing" at MOOCs. I didn't get credit but I got value.

The neurobiological limits of free will

Animals have limited, possibly no, choices. Biology and circumstances are their destiny.

I, being a human, have choices. They are limited but present. Like all humans, I have deep neural ruts (i.e., habits) that guide most of my thoughts and behaviors. They are not a permanently fixed boundaries; I can change them (slowly). I have the free will to choose to change those habits. It ain't easy (or quick) but possible.

Understanding the neurobiology can help. Knowing that simple repetition is more important than willpower. Knowing that old habits are easily cued, even after long periods of being dormant. Knowing that all brains have strong default systems, mostly around fear, that don't serve higher callings in the modern world. Knowing that there is never conscious access to unconscious processes, thus just trusting that providing "open space" will allow unconscious processes to manifest.

I'm humbled because I don't change even what is within my very limited ability. I "know" the rules but still don't play the game of change well.

Web 3.0 (possibly)

Web 1.0 connected ideas with webpages and hyperlinks.

Web 2.0 connects people with social media and mobile.

Web 3.0 could connect physical things with automation.

The Internet of Things (IoT), both the concept and the necessary features (e.g., infrastructure, tools, and protocols), are gaining momentum.

Each successive stage of the web relies on previous stages. The previous stage becomes the assembly language of the next stage. If static web content can be generated automatically, there is then engineering bandwidth to tackle dynamic web content.

The number of processes that can be automated continue to increase. We are now the cusp of the next generation of web automation - the cost of aggregating the information from the physical items, from consumer products to industrial processes, is dropping below the return-on-investment threshold.

The potential amount of data from this transition is staggering. The previous two versions of the web each redefined "big data." We are on the cusp of another inflection point. There isn't enough engineering bandwidth to make sense of this data with static / human-based systems.


The most promising direction is automated learning, aka things that get smarter the more you use them. Previous generations of technology were static. Excel or your browser doesn't get any better the more they are used. However, there is now software that learns as you use it and search algorithms can get smarter the more they are used.

Machine learning is the tool to leverage the promise of Web 3.0.

Video of my Data Science panel appearance

I was recently on a Data Science and Machine Learning for Social Good panel. I enjoyed sharing my experience working at BrightBytes. It was nice to talk with bright people that work on more than click-through rates.

(I'm 3rd from the left.)

The connection between broadsiding and blogging

On a recent visit to the National Archives, I was engrossed with a broadside display.

Broadsides are the blogs of their day, a blend of art and news.

Both have a lower barrier to entry compared to other respective contemporary formats (e.g., treatises or whitepapers). Both are designed to be ephemera. However, an individual exemplar sometimes resonants beyond its intended lifespan and audience. That long-term impact potential is greater for blogs. Since blogs are digital, they are searchable and sharable without limits. Since broadsides are printed, they are static and nonscabable. Their size and content is limited by the properties of the physical press, printing press of the broadside heyday most cheaply produced a single page.

The illusion of being ephemera makes both more accessible, thus encouraging writing (and publication). Even if the writing is intended to be disposable, more writing makes better writers.

Treating manuscripts like pieces of code

As I revise a manuscript, I find bugs (i.e., things that are not the way they should be). If they can be fixed in less than two minutes, then I immediately correct them. Otherwise, I stack and track via a bug list. My bug list has two contexts: full focus and brain dead.

Brain dead bugs can be fixed with minimal cognitive effort. For example, fixing the axis on a figure.

Full focus bugs require cognitive horsepower. For example, synthesizing previous research.

Sometimes sharp tools aren't needed for interesting problems

Sharp tools help at the edges of science and business. If your tools are dull, you simply cannot keep up. However, you probably do not need the Large Hadron Collider to make a discovery or a High-Performance Computing Cluster to make your business idea a reality.

Given the tools you have access to at this instant (which is probably more than most people had 5 years ago), how can you make a dent in the universe?

Infastructure over willpower

I have very little willpower and don't like use the little amount I do I have.

Instead, I choose to create an infrastructure that makes the "right thing the easy thing." I set a timer for stretch and perspective breaks during head's down work. I keep my computer and web browser "clean." They only show me the right thing at the right time.

I carefully construct that infrastructure to be right for the given context. Very few options (or none) in my physical or virtual environment gives me the freedom I'm looking for.

Drowning in (the) waterfall (of) research plans

Most researchers plan too much (if they plan at all). I have seen Gantt charts and reverse calendars for research projects that do not reflect reality by the time the laser printer spits them out.

By definition, you conduct research because you do not know everything. It is foolish to think you will not discover new information about your project or your process by "doing the work."

This is where Agile software practice can inform research practice. Set a quick iteration goal (e.g., collect pilot data or run primary statistical analysis). Then assess what was done and how it was done. Adjust course.

Are we running in the right direction?

How can we run faster?

Daily stand-ups wouldn't hurt either.

My focus on commonalities over differences

Humans are remarkably similar. As a species, we have relatively little genetic diversity[1]. There are cultural differences. But IMHO - our commonalities trump our differences.

I rather spend my limited time understanding the rules of humanity than the exceptions.

My scientific research focused on the basic process growing out the unique neurobiological constraints of humans. Much of psychology focuses on minor differences. The narcissism of minor differences is a sticky subject. Endless fascinating but has only a fraction of the power of unlocking our shared fundamentals.

---

1. Probably a result of a near extinction-level event.  ↩
   

Creating playgrounds

Everyday I create playgrounds. Some of them are physical, most of them are virtual.

I see who comes to play with me. If not enough of the right people want to frolic, I change the structure or the rules. It is easier to change virtual playgrounds, but there is something magic about exchanging molecules.

I don't stress if the playground is right the first time (or ever "perfect"). I baked-in continuous change with continuous improvement is a byproduct.

Working with visionaries

Visionaries are big dreamers. They are a driving force for change in the world. They are the spark that ignites the fire of revolutions.

They are also crazy makers. Their ideas are unbounded by external constraints. They overextend themselves and the people that work for them. They switch directions like the wind on top of Mount Everest.

I choose do not chase the wild ideas of visionaries. Ideas are real to them; Ideas are their world. I listen (and acknowledge) their world but always pause before I allow their idea-based world to influence my action-based world. I take time to weigh their ideas against the discrete, real-world actions steps necessary to manifest them.

Modern day mathematician

I have always wanted to be a mathematician (weird I know). Their job is playing with pure ideas and solving hard problems. All a mathematician needs is paper, a pencil, and a waste basket.

There is an old proverb - "A mathematician never soils his hands with calculations." First, ignore the gender bias. Second, in the time before computers that perspective made sense. Actual calculations waste precious time. Now that world view is limiting. Testing ideas, at any scale, is computational efficient. However if there is little (or no thought) for Big O, it is difficult to compare multiple correct methods.

I'm pragmatic in choosing between abstract calculations of run time and Monte Carlo simulations. I choose the quickest, best time estimation method for a given problem. But I always makes estimates. A modern day mathematician has to realize the constraints of reality on pure solutions.

Action-oriented reading

Just reading an item (and most likely forgetting it) is a low-value activity for me. Reading should be more than just providing the raw material for the cocktail party grist mill. I want to make a dent in the universe - make the world a better, more interesting place. I read as a means towards that end.

I ask myself:

• What will I do differently because of what I just read?
• How will I take what I have read and manifest it as visible, positive change?

Academic courting ritual

There is much to learned in graduate school and depending on your mentor you might not get the mentoring you need. Some mentors are human-sized lab rats, living in the lab and pumping out publication after publication. Other mentors are gregarious connectors, they know which other researchers have free cycles and available money (possibility more important). Both personality traits are useful in scientific career but the social bootstrapping is the one most frequently missing.

Part of a young scientists' maturation is developing the ability to work with collaborators. Those collaborators hopefully will grow into colleagues, people to join with for grants and suggest graduate students. This is the power of weak ties. The process starts with the academic courting ritual.

Academic courting ritual:

1. Make small talk
2. Find common connections
3. Discuss current projects
4. Exchange papers to read

Think optimal, not just possible

We live in amazing times. You can find out any fact or connect with almost anyone in the world on the device in your pocket. People then go out of their way to "prove" they do anything on their little screens. "I have a startup focusing on mobile strategy for Big Data analysis." Really? Just because you can doesn't mean you should.

Complex, creative work is faster and better on a big screen (preferable multiple).

Programming instead of math; statistics instead of calculus

Schools should be teaching computer programming instead of math^[1]. Math is a systematic, rigorous thought system that builds quantitative reasoning and problem solving skills. Programming taps into a similar set of skills but is more practical. The ability to write functional code will take the most people further than factoring polynomials. It could be as simple as Automator or Visual Basic for Applications (VBA) in Excel.

Schools should be teaching statistics instead of calculus. Very few people use the calculus taught in school. However probability, the fundamental basis of statistics, is a general life skill. Thinking probabilistically cracks open the door of possibility that our typical deterministic thoughts leave closed. With the increasing amount and power of data, a basic data analysis toolbox is more valuable than infinitesimals.

School should be preparing students for the world as it is and how it will mostly likely be. Far more people are programming than using math, and far more people are applying statistics than calculus.

---

1. By math, I mean algebra, geometry, and trigonometry. Basic arithmetic is a necessary skill for every functional citizen.  ↩
   

Fear-driven professional development

Knowledge workers are susceptible to nonclinical impostor syndrome, a lurking fear that they will be found out as a fraud and haven't earned what they have. Those feelings are heightened with streaming access to the best people in the field. It is easy to find someone who is more clever, more accomplished, a little more everything.

In turn, driving a frantic obsession with self improvement at work (with residual guilt for not doing enough). Instead ask, "How much better can I be in order to give back to the world?" - love-driven professional development.

On Writing (Science) Well

I'm working my way through On Writing Well by William Zinsser (hat tip to Merlin Mann). It is a simple, but not easy, approach to better nonfiction writing. It advocates boldness. Directly stating is better than obliquely suggesting. Active is better than passive.

This is very different from my default writing style for science and statistics. Standard science and statistics language is too often passive and awkward. While revising one of my current manuscripts, I have found the following peccadillos:

• “This can be seen when”
• “All procedures were approved by”
• “could potentially have”
• “can be described as”

Mindfulness is the first step towards change.