Protected: Air Freshener

This content is password protected. To view it please enter your password below:

Posted in Writing | Comments Off on Protected: Air Freshener

Prerequisite 1: Machine Learning

To get into any subject, there is a lot of jargon to wade through. I love simplicity but precise terminology makes things much easier. In this post I’m going to establish basic definitions for these terms, which are necessary for deep learning:

  • machine learning
  • model
  • supervised learning
  • prediction
  • cost function
  • function optimization
  • local minimum vs. global minimum

Everyone who reads this post has some idea of learning. What does it mean for a machine to learn?

In the machine learning context, to learn means:

to create the ability to predict from data.

The method used to predict the data, learned or otherwise, is called a model. When a machine learns a model, then machine learning has happened!

A model can be thought of as a function, a math function. If you know about functions, think of the model as a function F where F(data) = prediction.

An example of data, predictionmodel:

Suppose we have a month’s worth of weather data for Redwood City California, and we want to predict weather based on this data. The data is that in 30 days, it rained 3 days. So it rained 10% of the time.

Here is a very simple model: based on these 30 days of data, the probability of rain is 10%. This is a pretty low probability, so I predict no rain for all days. In my function language: the learning process based on historical data gave me a function F:

Learning Process (30 days of data: 27 no rain, 3 rain) = F = no rain

To elaborate on the function F:

F(any time) = no rain

If the weather continues to be the same, my model, the “no rain” model, has an accuracy of 90%. Impressive, isn’t it?

Despite the “no rain” model’s high accuracy (especially in times of drought), can we point out other limitations of this model? We will come back to this question. It is really a good chunk of the work of a scientist – deciding what to do with your model after you’ve computed it. This model might be obviously bad – but many models suffer from exactly the problem that this model does. Care must be taken!

Notice that very little learning took place to create this model. One calculation, the ratio of rainy days to all days, is used as the basis of all predictions. We can maybe say that the “learning” method used in this weather example is just extrapolation, or trending. And the computer didn’t learn it, we did. We decided the formula to use to predict the weather.

How do we get a machine to do the learning? How do we get a computer to build a model?

We give the machine some scaffolding, and program the machine to compute the details – the “walls and floors” of the model.

Since the machine is a computer and computers work with numbers, the scaffolding is a particular class of function, and the computer fits a “best” function of that particular type that matches our observed data.

The method used in almost all of machine learning is this: Estimate the cost of a mistake. Formulate it with variable parameters – use a template formula for your model function that has variable parameters. Minimize that cost with respect to those parameters – find the value of those parameters that gives you the least cost; find the “best” value of those parameters. That gives you the “best” function.

We will discuss this in more detail in another post.

Posted in Writing | Comments Off on Prerequisite 1: Machine Learning

PSA: What makes a good personal essay?

It’s college essay season again. As hard and annoying as they are to write, I’m a huge fan of the genre of which they are a sub: the personal essay. The personal essay, when well done, exploits tools of both expository and creative writing. Here are some excerpts from my heroes and some notes about why their personal essays are wonderful.

James Baldwin

Notes of a Native Son

On the 29th of July, in 1943, my father died. On the same day, a few hours later, his last child was born. Over a month before this, while all our energies were concentrated in waiting for these events, there had been, in Detroit, one of the bloodiest race riots of the century. A few hours after my father’s funeral, while he lay in state in the undertaker’s chapel, a race riot broke out in Harlem. On the morning of the 3rd of August, we drove my father to the graveyard through a wilderness of smashed plate glass.

James Baldwin is an amazing writer. This paragraph is an example of the power of simplicity. He has an unusual phenomenon to describe, the confluence of three huge events: the death of his father, the birth of a sibling, and a race riot. He just needs to make that clear. Being a master, he brings it all together at the end with enormous force. Note how that simple last sentence serves many purposes all at once: it evokes sight and sound (you can’t help but imagine the glass crunch beneath their wheels), it sets us up for a journey, describes the outer world of chaos, describes his inner broken heart.

If you have an amazing story to tell, start simply. Don’t let the writing overshadow the events. And bring in a concrete detail that makes it alive.

Virginia Woolf

The Duchess of Newcastle

‘… All I desire is fame’, wrote Margaret Cavendish, Duchess of Newcastle. And while she lived her wish was granted. Garish in her dress, eccentric in her habits, chaste in her conduct, coarse in her speech, she succeeded in her lifetime in drawing upon herself the ridicule of the great and the applause of the learned. But the last echoes of that clamour have now all died away; she lives only in the few splendid phrases that Lamb scattered upon her tomb; her poems, her plays, her philosophies, her orations, her discourses – all those folios and quartos in which, she protested, her real life was enshrined – moulder in the gloom of public libraries, or are decanted into tiny thimbles which hold six drops of their profusion. Even the curious student, inspired by the words of Lamb, quails before the mass of her mausoleum, peers in, looks about him, and hurries out again, shutting the door.

Woolf makes good use of variety in her sentences. She starts with a quote, then parallel inverted clauses, then parallel noun phrases. Concrete language grounds all the abstractions: from garish, eccentric, and chaste we go to ridicule, applause, a dying clamour, tomb, mould, decanters, thimbles, drops. And then concrete images of actions: quails, peers, looks about, hurries, shuts.

This first paragraph is inviting. I want to taste the liquid in one of those thimbles. I want to see if I agree with Lamb, or think he’s full of fluff. I want to be braver than the curious student.

Ursula K. LeGuin

I am a Man

I am a man. Now you might think I’ve made some kind of silly mistake about gender, because my name ends in A, or I own three bras, and I’ve been pregnant five times, and other things like that that you might have noticed, little details. But details don’t matter. If we have anything to learn from politicians it’s that details don’t matter. I am a man, and I want you to believe and accept this as a fact, just as I did for many years.

Hang on to your hat, this essay is going to take your for a wry and ironic ride. This is a great example of good use of shock value, sly insertion of philosophy and jokes, and good clear, varied sentence structure to boot. Note her grounding of abstract ideas into concrete imagery – what does it mean to be a woman? Do bras make a woman? What are details and what matters? She poses these questions in an entertaining way.

This is an example of a high-concept essay – using a bold set-up to explore many different kinds of ideas.

Ian Frazier

My First Summer Living in a Van

I had not expected that it would tick. As soon as the sun hit it in the morning—at 6 a.m. or so, in June, in northern Michigan—the metal would start to expand in the heat: tick…tick…tick. My first summer living in my van, in the Pigeon River campground near the town of Vanderbilt, I almost never succeeded in sleeping past dawn. And I was not prepared for how stuffy it got with the windows rolled up against the mosquitoes. I thought the overhead light, which had come loose and dangled by its wires above the rudimentary plywood bed and foam-rubber mattress that I slept on, would be enough light to read by. It wasn’t.

Great example of ‘in medias res’, starting right in with the situation at hand. Ian Frazier is a prolific essayist. He also knows how to tackle abstract ideas using concrete stories and images to connect to the reader and bring the subject home.

George Orwell

Some Thoughts on the Common Toad

Before the swallow, before the daffodil, and not much later than the snowdrop, the common toad salutes the coming of spring after his own fashion, which is to emerge from a hole in the ground, where he has lain buried since the previous autumn, and crawl as rapidly as possible towards the nearest suitable patch of water. Something — some kind of shudder in the earth, or perhaps merely a rise of a few degrees in the temperature — has told him that it is time to wake up: though a few toads appear to sleep the clock round and miss out a year from time to time — at any rate, I have more than once dug them up, alive and apparently well, in the middle of the summer.

This essay connects nature to politics, society, and philosophy in a wonderful way. Note again the sentence variety, concrete imagery, strong verbs. The pacing in this piece is slower than a college essay can be, but if you read it you will still be amazed at all the ideas Orwell manages to pack into the essay.


The examples above all show how great writers use:

  • sentence structure – they are correct, clear, varied; they borrow from spoken language when needed
  • concrete imagery – they don’t overdo it, but they bring it in to show rather than tell
  • ambition – they write about deep things, but do the work to get there. You’d have to read the full essay to see what I mean. They shoot for the fences.

In a college personal essay, keep in mind that it is not only a personal statement but a professional one. It makes a case for you as a student. Tell success stories, not sob stories. Don’t say anything you would not say in person, to someone’s face, in a job interview. Take a stand, of some sort. Whatever stand you take (bold, ironic, bitter, hopeful, witty, futuristic), make every word in your essay hang on that thread. If you read each of the essays above, they hang together.

Posted in Writing | Comments Off on PSA: What makes a good personal essay?

My baseline hero

In 1976, a major earthquake struck in Tangshan, China. Because many of the buildings had not been built to withstand earthquakes, and because search and rescue infrastructure at the time was poor, the death toll was large: the official estimate is about a quarter of a million people.

In The Good Women of China, I read about a woman whose experience was horrific. Her fourteen year old daughter was pinned from the waist down by concrete rubble from their apartment building. For over two weeks they waited for rescue, unable to move the rubble themselves. At first they were hopeful. The girl could talk and eat and drink. She wasn’t in much pain, as she couldn’t feel anything in the lower half of her body. In the massive surrounding destruction no help came at all. The girl lost consciousness and died, as her mother watched, hoped, waited, despaired. Second by second.

Here is why this mother is my baseline hero, the one I find myself comparing all other heroes to. She had lost everything in the quake: her family, her home. And what did she do? She got a live-in job at an orphanage, wiping noses, washing diapers, making oatmeal.


What’s so heroic about that? One, she saw an extensive problem that she had the skills to solve, and she went and worked on it. I’m sure there’s some line that says orphans are intrinsically compelling, after all, and much can be made of small tokens: occasional hugs, smiles, crayon drawings of hearts. But if these tokens are enough, why do so few people take care of orphans? No. Devoting her work time to them is a big thing, and when I read about how she went about it, there was no flavor of sentimentality or angels. It was pragmatism. The problem was large, right in front of her, and she could do something about it. She did. What it looks like to me is a data driven choice.

Two, in order to do that, she had to struggle past her own tragedy. She had good reason, after what she went through, to sit in a bar and “stick her neck out for nobody.”

The book described how her grief and horror never went away. Maybe outrage, too. She had regular nights when she could not sleep. But in the morning the work still had to be done, and she did it.

I can’t think of a purer example of someone who had to get over herself in order to accomplish what she set out to do. With very little help: without the crutches of fame or fabulous heights of remuneration. No honorary PhD of good-scout-logy. Not even a donated Vera Wang gown to wear on the red carpet. No red carpet.

Where I live, tech heroes are very popular. Arts and science heroes too. It’s also popular to spend a lot of time on passions and dreams. And a lot of what happens in the realm of philosophical discussion is about privilege.

For me, thinking about this woman from 70s Tangshan cuts a clean swath through narcissism and victimhood. She is an ultimate hero because what she has done with her life is so awesome, she shows the way. I want to be on her team.

Posted in Data Science | Comments Off on My baseline hero

What is deep learning?

Deep learning is a buzz phrase that refers to a branches of machine learning that involve multiple, interactive layers of nonlinear data transformations. It’s deep because there are multiple layers, and it’s deep learning because the layers can communicate with one another. Deep learning techniques have been growing in accuracy and success and are interesting to study in themselves.

As a data scientist, I’m drawn to deep learning because it involves some of my favorite things: multidimensional math, linear algebra in particular; cool algorithms; the chain rule; and surprises – which in the applied math world means “nonlinear effects”.

So my goal is to document adventures in deep learning in the next months, and make this blog a place where interested people can learn along with me.

Posted in Data Science, Machine Learning, Software Engineering | Tagged , , | Comments Off on What is deep learning?

How I learned to stop worrying and love computer programming

This gallery contains 1 photo.

As a go-getting engineering student, I chose my first university elective to be Computer Science 101. I didn’t like it. The lectures put me right to sleep – and I’m a lecture lover. The exams were not a problem. But … Continue reading

More Galleries | Comments Off on How I learned to stop worrying and love computer programming

My brutish love of poetry, part 4

Poetry can be dangerous. Yes, it can be pretty, pithy, wise, amusing. But it’s gotten me and countless others into trouble. My web site here is a pretty safe place and I’ve quoted some pretty safe poems. There are many … Continue reading

More Galleries | Comments Off on My brutish love of poetry, part 4

My brutish love of poetry, part 3

This gallery contains 1 photo.

Another gift of Canada is exposure to French Canadian poetry. If you know even a little bit of french, you can poke around at the edges. I am a fan of Anne Hébert and of her cousin Hector de Saint-Denys … Continue reading

More Galleries | Comments Off on My brutish love of poetry, part 3

My brutish love of poetry, part 2

This gallery contains 1 photo.

In 1920, Marianne Moore wrote a poem that begins: The Fish wade through black jade. You can immediately see that this poem plays with line breaks and structure and is not about meter and rhyme. Yikes, free verse. What is … Continue reading

More Galleries | Comments Off on My brutish love of poetry, part 2

My brutish love of poetry, part 1

I never studied poetry the way I studied other subjects. But I love it, poems stick in my head for years, I can recite a few, and sometimes reading a poem colors my thoughts for hours. Since I don’t know … Continue reading

More Galleries | 1 Comment