To get into any subject, there is a lot of jargon to wade through. I love simplicity but precise terminology makes things much easier. In this post I’m going to establish basic definitions for these terms, which are necessary for deep learning:

• machine learning
• model
• supervised learning
• prediction
• cost function
• function optimization
• local minimum vs. global minimum

Everyone who reads this post has some idea of learning. What does it mean for a machine to learn?

In the machine learning context, to learn means:

to create the ability to predict from data.

The method used to predict the data, learned or otherwise, is called a model. When a machine learns a model, then machine learning has happened!

A model can be thought of as a function, a math function. If you know about functions, think of the model as a function F where F(data) = prediction.

An example of data, prediction, model:

Suppose we have a month’s worth of weather data for Redwood City California, and we want to predict weather based on this data. The data is that in 30 days, it rained 3 days. So it rained 10% of the time.

Here is a very simple model: based on these 30 days of data, the probability of rain is 10%. This is a pretty low probability, so I predict no rain for all days. In my function language: the learning process based on historical data gave me a function F:

Learning Process (30 days of data: 27 no rain, 3 rain) = F = no rain

To elaborate on the function F:

F(any time) = no rain

If the weather continues to be the same, my model, the “no rain” model, has an accuracy of 90%. Impressive, isn’t it?

Despite the “no rain” model’s high accuracy (especially in times of drought), can we point out other limitations of this model? We will come back to this question. It is really a good chunk of the work of a scientist – deciding what to do with your model after you’ve computed it. This model might be obviously bad – but many models suffer from exactly the problem that this model does. Care must be taken!

Notice that very little learning took place to create this model. One calculation, the ratio of rainy days to all days, is used as the basis of all predictions. We can maybe say that the “learning” method used in this weather example is just extrapolation, or trending. And the computer didn’t learn it, we did. We decided the formula to use to predict the weather.

How do we get a machine to do the learning? How do we get a computer to build a model?

We give the machine some scaffolding, and program the machine to compute the details – the “walls and floors” of the model.

Since the machine is a computer and computers work with numbers, the scaffolding is a particular class of function, and the computer fits a “best” function of that particular type that matches our observed data.

The method used in almost all of machine learning is this: Estimate the cost of a mistake. Formulate it with variable parameters – use a template formula for your model function that has variable parameters. Minimize that cost with respect to those parameters – find the value of those parameters that gives you the least cost; find the “best” value of those parameters. That gives you the “best” function.

We will discuss this in more detail in another post.

It’s college essay season again. As hard and annoying as they are to write, I’m a huge fan of the genre of which they are a sub: the personal essay. The personal essay, when well done, exploits tools of both expository and creative writing. Here are some excerpts from my heroes and some notes about why their personal essays are wonderful.

James Baldwin

Notes of a Native Son

On the 29th of July, in 1943, my father died. On the same day, a few hours later, his last child was born. Over a month before this, while all our energies were concentrated in waiting for these events, there had been, in Detroit, one of the bloodiest race riots of the century. A few hours after my father’s funeral, while he lay in state in the undertaker’s chapel, a race riot broke out in Harlem. On the morning of the 3rd of August, we drove my father to the graveyard through a wilderness of smashed plate glass.

James Baldwin is an amazing writer. This paragraph is an example of the power of simplicity. He has an unusual phenomenon to describe, the confluence of three huge events: the death of his father, the birth of a sibling, and a race riot. He just needs to make that clear. Being a master, he brings it all together at the end with enormous force. Note how that simple last sentence serves many purposes all at once: it evokes sight and sound (you can’t help but imagine the glass crunch beneath their wheels), it sets us up for a journey, describes the outer world of chaos, describes his inner broken heart.

If you have an amazing story to tell, start simply. Don’t let the writing overshadow the events. And bring in a concrete detail that makes it alive.

Virginia Woolf

The Duchess of Newcastle

‘… All I desire is fame’, wrote Margaret Cavendish, Duchess of Newcastle. And while she lived her wish was granted. Garish in her dress, eccentric in her habits, chaste in her conduct, coarse in her speech, she succeeded in her lifetime in drawing upon herself the ridicule of the great and the applause of the learned. But the last echoes of that clamour have now all died away; she lives only in the few splendid phrases that Lamb scattered upon her tomb; her poems, her plays, her philosophies, her orations, her discourses – all those folios and quartos in which, she protested, her real life was enshrined – moulder in the gloom of public libraries, or are decanted into tiny thimbles which hold six drops of their profusion. Even the curious student, inspired by the words of Lamb, quails before the mass of her mausoleum, peers in, looks about him, and hurries out again, shutting the door.

Woolf makes good use of variety in her sentences. She starts with a quote, then parallel inverted clauses, then parallel noun phrases. Concrete language grounds all the abstractions: from garish, eccentric, and chaste we go to ridicule, applause, a dying clamour, tomb, mould, decanters, thimbles, drops. And then concrete images of actions: quails, peers, looks about, hurries, shuts.

This first paragraph is inviting. I want to taste the liquid in one of those thimbles. I want to see if I agree with Lamb, or think he’s full of fluff. I want to be braver than the curious student.

Ursula K. LeGuin

I am a Man

I am a man. Now you might think I’ve made some kind of silly mistake about gender, because my name ends in A, or I own three bras, and I’ve been pregnant five times, and other things like that that you might have noticed, little details. But details don’t matter. If we have anything to learn from politicians it’s that details don’t matter. I am a man, and I want you to believe and accept this as a fact, just as I did for many years.

Hang on to your hat, this essay is going to take your for a wry and ironic ride. This is a great example of good use of shock value, sly insertion of philosophy and jokes, and good clear, varied sentence structure to boot. Note her grounding of abstract ideas into concrete imagery – what does it mean to be a woman? Do bras make a woman? What are details and what matters? She poses these questions in an entertaining way.

This is an example of a high-concept essay – using a bold set-up to explore many different kinds of ideas.

Ian Frazier

My First Summer Living in a Van

I had not expected that it would tick. As soon as the sun hit it in the morning—at 6 a.m. or so, in June, in northern Michigan—the metal would start to expand in the heat: tick…tick…tick. My first summer living in my van, in the Pigeon River campground near the town of Vanderbilt, I almost never succeeded in sleeping past dawn. And I was not prepared for how stuffy it got with the windows rolled up against the mosquitoes. I thought the overhead light, which had come loose and dangled by its wires above the rudimentary plywood bed and foam-rubber mattress that I slept on, would be enough light to read by. It wasn’t.

Great example of ‘in medias res’, starting right in with the situation at hand. Ian Frazier is a prolific essayist. He also knows how to tackle abstract ideas using concrete stories and images to connect to the reader and bring the subject home.

George Orwell

Some Thoughts on the Common Toad

Before the swallow, before the daffodil, and not much later than the snowdrop, the common toad salutes the coming of spring after his own fashion, which is to emerge from a hole in the ground, where he has lain buried since the previous autumn, and crawl as rapidly as possible towards the nearest suitable patch of water. Something — some kind of shudder in the earth, or perhaps merely a rise of a few degrees in the temperature — has told him that it is time to wake up: though a few toads appear to sleep the clock round and miss out a year from time to time — at any rate, I have more than once dug them up, alive and apparently well, in the middle of the summer.

This essay connects nature to politics, society, and philosophy in a wonderful way. Note again the sentence variety, concrete imagery, strong verbs. The pacing in this piece is slower than a college essay can be, but if you read it you will still be amazed at all the ideas Orwell manages to pack into the essay.

Takeaways

The examples above all show how great writers use:

• sentence structure – they are correct, clear, varied; they borrow from spoken language when needed
• concrete imagery – they don’t overdo it, but they bring it in to show rather than tell
• ambition – they write about deep things, but do the work to get there. You’d have to read the full essay to see what I mean. They shoot for the fences.

In a college personal essay, keep in mind that it is not only a personal statement but a professional one. It makes a case for you as a student. Tell success stories, not sob stories. Don’t say anything you would not say in person, to someone’s face, in a job interview. Take a stand, of some sort. Whatever stand you take (bold, ironic, bitter, hopeful, witty, futuristic), make every word in your essay hang on that thread. If you read each of the essays above, they hang together.