Archive | Lit Notes RSS feed for this section

Measure your words with text mining Python scripts

Python text mining scripts, count your words

Ursula K. Le Guin said she wrote 40 stories before the first of her stories was published. This is a collection of her first published stories.

Text mining scripts in Python

I wrote a set of Python scripts to run numerical analysis on my writing. You can gain valuable insight into your writing by measuring your words. The measurement of your writing can be part of your writing practice, and sets the basis for creating tests when validating hypothesis about your writing.

You can find the text mining scripts in my GitHub repository. The repo contains the following scripts:

  • Concordance (word count)
  • N-Gram (word pair count)
  • Entity extraction (keyword count)
  • Search engine keyword analysis (scores keywords)
  • Parts-of-Speech (part-of-speech frequency)
  • Reading level (reading grade level)

These scripts were partly inspired by a job I had as a social media analyst in 2008. And the desire to create this set of scripts is one of the reasons I started learning Python. Social media analysis mixes two different disciplines: social network analysis and text mining. Social network analysis is the kind of thing that epidemiologists use to find patient zero. For my job, I was using network maps to track who was talking about products and who was listening to them talk about products. For example, I looked at who was talking about salsa. To quickly assess what they were talking about I used some crude forms of text mining. My methods at the time typically involved Excel.

These scripts use the Natural Language Toolkit (NLTK) which has been around for a while.

GitHub Repo – Measure words

Count words

I have always counted the words in my writing, or in a handwritten journal, the time.

When I was sixteen years old and facing what seemed an essential choice of what I wanted to do with my life, this doing seemed more than a way of making money, but a choice that had to be a vocation. In my sophomore and junior year in high school, the school gave us a military skills test and career tests. I think both tests determined I should be a clerk.

I had planned on going to school to become an electrical engineer. In my family at the time the only white color jobs anyone had ever had been as engineers. I went to school were some of the parents were engineers for Boeing and some of the parents worked in the factories assembling airplanes. My grandfather had worked in a factory working on airplanes. My other grandfather worked on nuclear submarines. My father and uncles, however, had driven cabs and worked in restaurants. They weren’t chefs, but worked in the kitchen. And the thing about restaurant work is that no one thought of themselves a chef who worked in a Seattle dinner preparing Crab Louie. They thought of themselves as people who lived their lives and paid the bills by working in a Seattle diner preparing Crab Louies.

The engineers, however, identified themselves as engineers as if being an engineer was an existential condition. In the spring of sophomore year contemplating a profession of being an engineer where I would be an engineer rather than a guy who works on engineering to pay the bills, I decided if that as the deal I would rather be a writer.

I had no idea what this meant. Since engineers could earn a living in the state of being an engineer couldn’t writers get buy by being in a state of being a writer?

I didn’t really realize at the time that being a writer was more like being a clerk, and I was just affirming the effectiveness of the vocational tests.

I had a vague notion that a writer was someone who wrote and somehow things like housing and food were not that big of a deal. They just came with the gig. I may have been basing this on Jack Torrance from The Shining or Garp from The World According to Garp. I had three ideas that ended up being helpful to me.

One idea was that I had to write every day and that I was beginning from nothing and would have to learn by writing if I wanted to be a writer.

The second idea was that I had to finish stories. Garp wrote a story a month while in high school. And I had read in the introduction to a collection of Ursula K. Le Guin’s firs stories, The Winds Twelve Quarters that she had written stories as a regular practice and sent them out. She wrote 40 stories before she was published.

The third idea was that I had to send my finished stories to magazines to get published. This meant I had found out where these things were and who to send them to.

Every night beginning that spring in 1987 I sat down to write. I thought about my work like it was a homework assignment, and so learned I could write 500 words with some degree of concentration. Even 500 words meant hat in a week I had a number of words that indicated a length of a story.

In a year I had finished 10 stories. In two years I had finished about 20 stories and written what I thought was a novel which was about twenty thousand words. One of the surprising aspects of this habit was that it didn’t require that much time. I could write 500 words in less than an hour. After I learned to type, I could write that in less than half an hour.

I kept writing when I enlisted in the Army Reserve and went through Boot Camp and skills training at Fort Sam Houston. I didn’t have time to write in Boot Camp, but at Fort Sam, the base library had typewriters I could use and I bought onion skin typing paper and typed on the IBM Selectric typewriter they had, 500 words or more. And then learned that onion skin is not good to type on because the ink flakes off. So I retyped my stories.

Word count was a familiar metrics to me. It was like miles are to a long distance runner, or laps to a swimmer.

I thought of the count of finished stories as proof that I was progressing toward being a writer thinking there would be a state change at some point. I would be published, and thereafter be a writer.

I also sent stories out. I found The Writer’s Digest Writer’s Market. I learned to send a story with a return postage, and began to collect rejection slips. I expected rejection slips at first and then became used to it. Early on in 1988 I got a shock when I sent a story to a local magazine edited by writer I had read about in The Seattle PI, Jessica Amanda Salmonson. She sent me back a letter and said something to the effect that based on the strength of the title of my story she had read my entire manuscript. And this had been her mistake. She suggested I get serious psychological help as quickly as possible not only for my own safety by the safety of everyone around me. My story had been called, “Leave Shatter Like Skulls.” I was reading a lot of L. Sprague de Camp, Robert Howard, Micheal Moorcock, and HP Lovecraft at the time. I was thrilled that my story had been read and that it has struck a nerve.

Based on this letter that saw my writing not as writing but as the symptom of a deranged mind I kept at for years. As a college freshman in 1992 I won a prize but not publication in STORY magazine in a year that saw a writer named Benjamin Anastas from the University of Iowa MFA program winning the first prize. The next year I published a story the Bellevue College magazine Arnazella.

It was working in that it was less of a state change from not a writer to being a writer. Being a writer was more like being a runner. While actively running, I am a runner. While putting in a regular word count, I am writer. Jack Torrance is probably a good model. I think most people may think of Torrance as a proxy for Stephen King. And in terms of making a living as a writer, who wouldn’t want to be Stephen King? But in fact it is much more like Torrance putting in words in the lobby of the Overlook Hotel, just be nice to your family and don’t go to Room 237.

Comments { 0 }

Resist the Hivemind, thoughts on the Jack Straw Writers Program

2007 Jack Straw Writers

Nine years ago I was pleased to work with the following group of writers at the Jack Straw Writer’s Program: Doug Nufer, Anna Maria Hong, Susan Landgraf, Cheryl Strayed, Charles Potts, Corrina Wycoff Kathryn Trueblood, Laurie Blauner, Vis-a-Vis Society(Sierra Nelson and Rachel Kessler), Willie Smith, Howard W. Robertson, and Molly Tenenbaum.

In 2007, I was a curator for the Jack Straw Writer’s Program, a program created by Rebecca Brown and Joan Rabinowitz, and last night went to a reading at Jack Straw and it was kind of comforting (considering) to hear and see many local writer’s able to work, publish, and exist in the Pacific Northwest. I wrote the following essay in response to the question about curating the series and that every writer I was able to listen to them then, and they remain writers I listen to now even I haven’t been in touch with them for a long time. You can find audio of that year and following years at Jack Straw.

Resist the Hivemind

from the Raven Chronicles new issue “Celebrating 20 Years of the Jack Straw Writers Program, 1997-2016

On Facebook, I often read appeals to “Hivemind.” They write, ”Hivemind, can you tell me…” They do it without apology, as if all of our individual capacities as thinkers are reduced to a kind of communal processing capacity. We are the mental equivalent of ants. Writers resist this conception of thought. Writers who eschew cliché, doggerel, and sentimentality strike out toward the strange wilderness of what they think. When they are deep into what they really think and how they think their alien thoughts, their written or spoken work provokes me as a person. I can recognize myself in them, but also recognize someone who is not me. A book or a poem inevitably provides relief from the incessant pressure of my own presence.

A community somehow levels the progressive nature of the written word. It joins us into a structure with conventional standards of decorum and the watchful guidance of our fellow, polite, thinking ants in the Hivemind.

In late 2006, around the time that Facebook was opened to everyone over the age of 13, I was asked to be a curator for the Jack Straw Writers’ program. This allowed me the chance to listen to and engage with a collection of writers who could offer access to their interior thoughts. I felt myself drawn toward writers (or in the case of the Vis a Vis Society, a pair of writers) who adhered directly to that. Willie Smith embodies this urge. Willie will be the first to flog his writing with the communal standards of the Hivemind. And yet, year after year, he is incapable of bottling up his urges, confessions, and lurid suburban Cold War tantrums. I have had coffee or drinks with some of these writers, but I don’t claim to know them. We are not part of a physical community. Yet, we are a gathering of individuals who had cultivated, and continue to cultivate, a method of capturing our inner voice. I was pleased to hear how they read their work in 2007 and hear them speak aloud their inner voices. Nearly ten years later with Facebook used by 13% of the world’s population, they are still thankfully engaged in their work of writing as singular voices and not as part of the Hivemind.

Comments { 0 }

Literary Fiction is the Neo-Con Genre

a human face

Humans have a face.

It is odd to me how conventional thought and identity are represented in fiction. Most literary magazines and most literary fiction generally present a highly conventional sense of identity on the part of the humans that are in the stories. These humans stream-of-thought sounds similar (to us). The way they interact with the world is similar (to us). Even the larger structures such as plot assume certain motivations and actions (that we can relate to). As readers we expect these conventions to be in place.

Anyone who reads I suspect is either fitting their encounter with actual people into these conventional molds, or the are, as I am, happily confused by the strangeness of other people. In my case fiction, even naturalistic fiction, is as realistic as high-fantasy. The sympathetic narrator is as alien to me as an elf. Continue Reading →

Comments { 0 }

Response to the Words, Writers Should …

Do you disagree that writers should read a lot, that it’s good to spend time with one’s writing alone without oversharing, that writing students should meet their deadlines, and that it’s good to write with the reader’s enjoyment in mind?

Well it is good we don’t need to agree with each other, because I don’t agree. Mainly, it boils down to the concept of need to or should. I guess I don’t really see need to or should really be relevant to me as a writer. Who is another writer to tell me how I should or need write? And conversely who am I to tell them what they should or shouldn’t do. I’ll read (or not read) their writing.

On the surface what you are saying here (aside from need to/should) sounds reasonable, but they are not things I agree with.

angel-at-my-table-janet-frame

Kerry Fox as Janet Frame in “Angel at My Table.” The last scene of the movie is to me the scene of the writer at work. But then, so is someone with a laptop in a room full of other people typing.

Writers do not need to read a lot. If it were at all true that they did, then the widely read would often be great writers. (and the under-read poor writers; and how in the hell could Cicero or Lucretius be good writers at all considering the scant number of book in existence at the time?) I have experienced that the the widely read are almost always insufferable unless they just love books and read a lot of trash. For some reason I love talking to someone with a super sweet tooth as a reader. Consumers of junk for some reason are infectious when talking about lit. Personally I read a lot but also feel under read at all times.

Writing can be done in an iterative, constantly sharing confessional loop just as it can be done completely alone. This is one of the things I found fascinating about Tao Lin, and I find fascinating about my daughter’s generation. My daughter writes constantly. She is a fantastic writer. Her writing always includes feedback; she is always writing to an audience. I have been learning a lot about how to create a writing practice that includes this type of feedback. In my day job, where I am a writer, the good writing is done in short bursts that is constantly feed through a feedback loop. Like you, I read about the writer in their garrett, and when I went to school the model of production as a writer was one where the writer alone sat with their thoughts and wrote. When I started writing I read in the introduction to Ursula K. LeGuin’s collection the Winds Four Quarters that she wrote 40 stories before she published her first one, and so I spent the first five years of my writing doing just that; writing stories trying to get that number of 40 stories. My first book took 8 years to write and I did nearly all of that alone. (Like most writers I wrote manuscripts that will never see the light of day.) One book took 13 years to write and before it was ready to send out I didn’t show it to anyone. I like working alone. For me it is how I work; but I can see that is not the only way to work.

Continue Reading →

Comments { 0 }

Text Engines and the Future of Writing

I posted my design and speculation about how a “future” word processor will change composition at the PLUMB Blog.

The simple word processor remains a relic in our operating systems. Microsoft Word has bolted some concessions to the wired world into it — you can right-click and look up a word in Wikipedia — but the essential metaphor, a sheet of paper, harkens back to typewriters, correction tape, and postage stamps.

The old word processor should be thrown out. Do not even consider it. We don’t read on paper and haven’t been using it write for nearly a generation. What would the new word processor be like? And how might this new word processor change our conception of composition?

Click here for the entire thing.

Comments { 0 }

Rumpelstistkin, Pen Names, and SDO at PLUMB Blog

I posted some thoughts on the uniqueness of author names given Google and the importance of having a name that can be found on the internet if you think potential readers may enter your name into Google to find your books.

Writers have often deliberated over their names. They might take up pen names to protect their identity or names to conceal their gender. Lewis Carroll is really Charles Lutwidge Dodgson. George Eliot is really Mary Ann Evans. It is less common I think these days to do this. I imagine that sometime soon writers will consider the unique string (the sequence of characters) to create a unique online handle. A Google alert for the writer Fred Smith would be useless among the hundreds of Smith’s busy posting blog entries and photos to Flickr. A writer named, Dog11&Tree, would at least have the knowledge that she is the only one with that name on the Web.

Click here for the full post. Thanks.

Comments { 0 }

Where Do Ideas Come From? at Plumb Blog

I posted a short essay in response to two questions that always  seems to be asked of fiction writers: “What percentage of your work is fiction?” (100% — it’s fiction) and “Where do your ideas come?” from:

The phrase, “Don’t get any ideas,” is of course, impractical. When this is said to me, I already have many ideas and just the breath issuing the phrase gives me more ideas. “Don’t think of an elephant,” gives you some ideas about elephants. The phrase “don’t get any ideas about elephants,” is probably too filthy to contemplate.

Click here to read the entire thing at Plumb Blog.

Comments { 0 }

On Blurbs and Laurie Blauner’s Instructions for Living

Instructions for Living by Laurie BlaunerI recently read Blauner’s brand new and excellent novella Instructions for Living and wrote this blurb. I wonder if there is anyone who really excels at blurb writing? When I had my first book accepted for publication, I had to go hat in hand to authors I admired hoping they would say something. Chris Offutt who I had corresponded with on Salon’s Forums and whose work I really admire agreed to write something if he liked what he read. So I sent him the first part of the book. (He said he didn’t want to see the whole thing.) He sent me back a great blurb but also lamented the whole process. Who was he, he wanted to know. A good blurb would be from someone like Flannery O’Connor or John Cheever, and they were dead. I kind of feel in a similar vein about my own endorsement of a book. But as a writer I realize blurbs are about as important as jacket art — they come to represent the book and if they are accurate they help form an idea about where the book fits into the larger flow of books and words about books. I suppose in some way blurbs are a kind of protection for a book, like a jacket. The author is getting someone to create a short representation for the book and to vouch for the book. And in this case, I totally vouch for Blauner’s book. It is a chilling and increasingly relevant story about living in a bombed-out and repressive time.

Here is my blurb:

In sparse language, Laurie Blauner teases out the beauty of resisting the inevitable: the end of love, the end of life, the end of the world. At the core of her book, Instructions for Living, there is nothingness. The novella is a testament to Blauner’s remarkable skill; she has created a narrative that dissolves rather than advances. This is a chilling story appropriate for a world of pirate corporations, international gangs, and secret police forces.

You can buy the book here.

Comments { 0 }

Plumb Blog

I just posted my first entry in a group blog, Plumb Blog. I am pleased to be among such excellent company including Matt Baker, Charles Dodd White, Lavinia Ludlow, Cynthia Reeser, and others.

One of the paradoxes as a student of creative writing was the number of times I heard my teachers declare that writing cannot be taught. In what other profession than writing could a professional declare that the reason for his employment, the teaching of this vocation, is not possible? At the time, I wondered why I was paying this or that teacher’s tuition. And yet despite their best attempts not to teach me writing, I did learn how to write stories and novels from these instructors.

Click here to read the full entry.

Comments { 0 }

Ben Marcus, Not an Experimental Writer!?!

Right. Ben Marcus is not serious about not being an experimental writer. Come on, HTML Giant… keep up with the changes.

Comments { 0 }