Archive | Misc RSS feed for this section

Measure your words with text mining Python scripts

Python text mining scripts, count your words

Ursula K. Le Guin said she wrote 40 stories before the first of her stories was published. This is a collection of her first published stories.

Text mining scripts in Python

I wrote a set of Python scripts to run numerical analysis on my writing. You can gain valuable insight into your writing by measuring your words. The measurement of your writing can be part of your writing practice, and sets the basis for creating tests when validating hypothesis about your writing.

You can find the text mining scripts in my GitHub repository. The repo contains the following scripts:

  • Concordance (word count)
  • N-Gram (word pair count)
  • Entity extraction (keyword count)
  • Search engine keyword analysis (scores keywords)
  • Parts-of-Speech (part-of-speech frequency)
  • Reading level (reading grade level)

These scripts were partly inspired by a job I had as a social media analyst in 2008. And the desire to create this set of scripts is one of the reasons I started learning Python. Social media analysis mixes two different disciplines: social network analysis and text mining. Social network analysis is the kind of thing that epidemiologists use to find patient zero. For my job, I was using network maps to track who was talking about products and who was listening to them talk about products. For example, I looked at who was talking about salsa. To quickly assess what they were talking about I used some crude forms of text mining. My methods at the time typically involved Excel.

These scripts use the Natural Language Toolkit (NLTK) which has been around for a while.

GitHub Repo – Measure words

Count words

I have always counted the words in my writing, or in a handwritten journal, the time.

When I was sixteen years old and facing what seemed an essential choice of what I wanted to do with my life, this doing seemed more than a way of making money, but a choice that had to be a vocation. In my sophomore and junior year in high school, the school gave us a military skills test and career tests. I think both tests determined I should be a clerk.

I had planned on going to school to become an electrical engineer. In my family at the time the only white color jobs anyone had ever had been as engineers. I went to school were some of the parents were engineers for Boeing and some of the parents worked in the factories assembling airplanes. My grandfather had worked in a factory working on airplanes. My other grandfather worked on nuclear submarines. My father and uncles, however, had driven cabs and worked in restaurants. They weren’t chefs, but worked in the kitchen. And the thing about restaurant work is that no one thought of themselves a chef who worked in a Seattle dinner preparing Crab Louie. They thought of themselves as people who lived their lives and paid the bills by working in a Seattle diner preparing Crab Louies.

The engineers, however, identified themselves as engineers as if being an engineer was an existential condition. In the spring of sophomore year contemplating a profession of being an engineer where I would be an engineer rather than a guy who works on engineering to pay the bills, I decided if that as the deal I would rather be a writer.

I had no idea what this meant. Since engineers could earn a living in the state of being an engineer couldn’t writers get buy by being in a state of being a writer?

I didn’t really realize at the time that being a writer was more like being a clerk, and I was just affirming the effectiveness of the vocational tests.

I had a vague notion that a writer was someone who wrote and somehow things like housing and food were not that big of a deal. They just came with the gig. I may have been basing this on Jack Torrance from The Shining or Garp from The World According to Garp. I had three ideas that ended up being helpful to me.

One idea was that I had to write every day and that I was beginning from nothing and would have to learn by writing if I wanted to be a writer.

The second idea was that I had to finish stories. Garp wrote a story a month while in high school. And I had read in the introduction to a collection of Ursula K. Le Guin’s firs stories, The Winds Twelve Quarters that she had written stories as a regular practice and sent them out. She wrote 40 stories before she was published.

The third idea was that I had to send my finished stories to magazines to get published. This meant I had found out where these things were and who to send them to.

Every night beginning that spring in 1987 I sat down to write. I thought about my work like it was a homework assignment, and so learned I could write 500 words with some degree of concentration. Even 500 words meant hat in a week I had a number of words that indicated a length of a story.

In a year I had finished 10 stories. In two years I had finished about 20 stories and written what I thought was a novel which was about twenty thousand words. One of the surprising aspects of this habit was that it didn’t require that much time. I could write 500 words in less than an hour. After I learned to type, I could write that in less than half an hour.

I kept writing when I enlisted in the Army Reserve and went through Boot Camp and skills training at Fort Sam Houston. I didn’t have time to write in Boot Camp, but at Fort Sam, the base library had typewriters I could use and I bought onion skin typing paper and typed on the IBM Selectric typewriter they had, 500 words or more. And then learned that onion skin is not good to type on because the ink flakes off. So I retyped my stories.

Word count was a familiar metrics to me. It was like miles are to a long distance runner, or laps to a swimmer.

I thought of the count of finished stories as proof that I was progressing toward being a writer thinking there would be a state change at some point. I would be published, and thereafter be a writer.

I also sent stories out. I found The Writer’s Digest Writer’s Market. I learned to send a story with a return postage, and began to collect rejection slips. I expected rejection slips at first and then became used to it. Early on in 1988 I got a shock when I sent a story to a local magazine edited by writer I had read about in The Seattle PI, Jessica Amanda Salmonson. She sent me back a letter and said something to the effect that based on the strength of the title of my story she had read my entire manuscript. And this had been her mistake. She suggested I get serious psychological help as quickly as possible not only for my own safety by the safety of everyone around me. My story had been called, “Leave Shatter Like Skulls.” I was reading a lot of L. Sprague de Camp, Robert Howard, Micheal Moorcock, and HP Lovecraft at the time. I was thrilled that my story had been read and that it has struck a nerve.

Based on this letter that saw my writing not as writing but as the symptom of a deranged mind I kept at for years. As a college freshman in 1992 I won a prize but not publication in STORY magazine in a year that saw a writer named Benjamin Anastas from the University of Iowa MFA program winning the first prize. The next year I published a story the Bellevue College magazine Arnazella.

It was working in that it was less of a state change from not a writer to being a writer. Being a writer was more like being a runner. While actively running, I am a runner. While putting in a regular word count, I am writer. Jack Torrance is probably a good model. I think most people may think of Torrance as a proxy for Stephen King. And in terms of making a living as a writer, who wouldn’t want to be Stephen King? But in fact it is much more like Torrance putting in words in the lobby of the Overlook Hotel, just be nice to your family and don’t go to Room 237.

Comments { 0 }

Adam’s Replication Process

while-adam-sleeps-eve-is-formed-from-one-rib-late-12th-cIf you were the last one on Earth, you could replicate yourself using Adam’s Replication Process: pull a rib from your body and make an opposite-gender clone of yourself. You could then have sex with your clone. If incest produces messed-up genes, I imagine then procreating with your cloned rib must make for some exceedingly messed-up kids. And mind you, to continue the human race, these kids would have to reproduce with each other. So the human race would continue, but it would be a very genetically messed up, in bred version of itself.

Comments { 0 }

Hippie Tradition: Oxymoron?

It often seems that not only were the hippies in their twenties hostile to folks who was older (don’t trust anyone over thirty) but they also applied this hostility to anyone who was younger. How often have you heard an old hippie say, “You weren’t theeeere, man.” How would it be possible given this stance to create a tradition or legacy? I think that such a project is antithetical to the hippies. They probably see the phrase, “hippie tradition” as an oxymoron. I think this might account for the popularity among hippies of the quote attributed to both Grace Slick, and Robin Williams, and whose origin has been appropriately be lost, “If you remember the sixties you weren’t there.” And yet the hippies managed to have a lot of children and even managed to get very, very old.

A search in Google for the phrase “hippie tradition” actually turns up the term in active usage.

Comments { 1 }

My Short Career as a Radical Shoe Designer

Last summer I traveled to Lawrence KS. While my wife and I were there she took me to see a fashion show and I was stuck by the freewheeling, radical, and whimsical nature of the designs. Once I had enough to drink I was no longer bothered by the fact that I was wearing a wool jacket that I bought sometime last century at the GAP. I began to think, this is something I can do. I can do this. If I can get someone to build and design my dreams, I can wear my turn of the century GAP jacket and do what I want — people will mark it down as eccentricity. They will say, he is a shoe designer. Of course, I won’t wear the shoes.

 Shoe Design Template

Shoe Design Template

My step was to figure out how to draw a shoe. I would skip any kind of training in nonsense such as fabric, leather, or orthopedic considerations. These were limitations to the imagination. If I could draw it, then it could be fabricated. I could pay someone to wear it, someone to photograph it, and then I would beat off to the global production system. Definitely green production. Definitely with workers rights in mind. Made in Omaha.

I created five radical shoe designs stealing, er rather, creatively-inspired by the basic form of the Christian Louboutin shoe-boot.

My initial presentation didn’t garner any interested manufactures. But even Thomas Edison and Christian Dior had slow starts. I present to you: my five shoe designs.
Continue Reading →

Comments { 1 }

Use Homophoner to Infuse Your Text with Homophones

This is great. It works very smoothly provided you insert ASCII text. It is simple. It is, well, evil. This Web-based tool will change your text into a homophone nightmare. Sadly, for me, I can’t really tell. I’m only unsettled, slightly:

Early inn thee film, thee young man back-from-the-war leans inn too kiss his brother’s wife. Thee young man back from thee wore is vary young. He has a long beard withe split ends, butt his skin is ruddy, and his lips are read and his teeth are thick and strong. His brother and wife are even younger and inn thee parlance of Hollywood, yew wonder how young they can bee? Are they still teenagers? wee don’t no thee actors inn this film, a search inn IMDB yields credits and thee credit themselves point too credit films butt they are things yew halve knot scene. thee young man kisses his brother’s wife wile his brother is having a tantrum inn thee forest. thee younger brother stands inn thee middle of a field of wiled daisies withe a stick.

This text was homophonerated at
This text can be unhomophonerated at here.

Comments { 0 }

Fur and People

I wanted to describe two random images. I don’t know the story about them, but they came from a widget that grabs images from The Bible. A long time ago when studied art history, I kind of knew these stories because all of these painters and drawers painted the same stories from the Bible over and over again.

Giotto (top) and Durer (bottom)

Giotto (top) and Durer (bottom)

The Last Judgment (Detail of Hell 2) : Giotto : Three blue-fur covered creatures with goat fur or greasy feathers, furry bird legs, and the feet of doves circle two flesh-colored figures, a man and a woman. Perhaps it is the fabric in the man’s hand, the contrast of cloth to skin? These are not Adam and Eave, I think. They seem very much to understand they are naked. The man appears old, with streaks of grey in his hair. He is shaved, naked, his skin nude of even hair. He has a barrel chest and oddly skinny arms bent into triangles. He holds a pink satchel that appears that it could function as a purse, but given that neither he nor the woman wears any clothes the bag is more of a symbol really than even a purse. That is the symbols in the picture seem more meaningful than the actual objects the represent. The man for instance is a human male figure warped into the figure of a triangle – despite the damage to this triangle does to his arms and the angles of his arms and the general naturalistic structure of his body. His figure is made into interlocking triangles despite the resistance, the reality, of tendons and bone. One of the furry men holds a brush, but instead of fibers coming out of the top of the brush four long nails come out of the brush. The furry man is pushing the brush into the back of the naked male figure as if he were brushing his back, or scratching his back, so that the nails go into the man’s back. There isn’t any blood on the naked man. Although streaks do cover his back. They don’t look like streaks of blood. I do not know what substance they are represent. The man is also not howling. His expression is of a person having his back brushed. He is staring into the mid-distance, holding his pink bag, while the furry creature sticks the brush of needles into his back. Maybe it is a kind of acupuncture? The figure of the woman is that of a man. It isn’t a woman at all. It isn’t even a very good man in painterly drag. The figure of the woman is the same as the man – oddly distorted and skinny legs and arms. She has slightly more volume in her legs. She has a pelvic mound that buried in fur (most likely to conceal her penis). She has a round, fully packed stomach. Perhaps her stomach is intended to signify a pregnant stomach? Instead it looks like a man’s stomach who drank a lot of beer and the beer is still currently in the stomach. She has two pointy male-style breasts. They are breasts, though, with defined nipples in their down hanging line. I saw a documentary about weight lifters. Male weight lifters who take steroids sometimes develop breasts. These male breasts are mostly muscle with a tiny, jiggling terminal of fat. In the weight room, this documentary said, these are called bitch tits. They are a sign that the weight lifter is on roids. This male figure, who is supposed to be a woman, has these weight lifter roid breasts as well. The other signifier that she is a woman is that her very male face is paler than her other nude partner, and her long hair a uniform golden color is neatly combed. The man’s hair, in contrast, is wild and grey. A strange green dragon nearly the same height as the supposedly female figure clings to her. She has a bloody spot on her check. The dragon thing is green and articulated and has the head of a ferret. How it manages to hold onto the woman’s body is a mystery. Perhaps it is sticky feet or the figures are made of metal and a magnet holds it to her?
Continue Reading →

Comments { 0 }

My Poor Handwriting Is a Nice Font

I used this great, free program online, YourFonts, to convert my handwriting into a true-type typeface. I spent a long time saving up for a copy of Fontographer — it cost about three hundred bucks — and drawing matrices with its crude vector editor in 1998. It took me months to come up with something that looked twice as cracked as the typeface I made in about six minutes using YourFonts‘ free Web site.  All I had to do was print out their template, write my handwriting, scan my template, and upload it, download the font, and install it. It’s amazing. I have no idea why they are doing this unless they are collecting some kind of massive compendium of folk typefaces for a writing recognition program or something. But it is well worth the ten minutes to create a typeface.

Continue Reading →

Comments { 0 }

The Future of Hugo House (Not that the Board or really anyone else really cares)

RE: Not With a Bang, But a Whimper

Dear Ryan Boudinot,

I concede that my sources, Jason Epstein writing for the New York Times and the National Endowment For the Arts are probably flawed due to the vagaries of low-paid fact checkers and overworked analysts. We’ve all been there.

The details of our exchange have become too complex to deal with in the confines of a Web forum.

It has come down to this. You and me. The future of the Seattle writing community clearly, certainly, depends on us and our ideas about outreach programs at Richard Hugo House.

I concede, too, that perhaps a business minded approach is appropriate considering we are talking about an arts organization with a budget and employees and things.

In this spirit, I suggest we resolve our difference in the time honored traditional of all business minded people: dueling PowerPoint presentations outlining the potential futures of Richard Hugo House. In the yawning vacuum of Lyall Bush’s mysterious departure, sense must be made, preferably in three word bullet points.

I suggest we meet in appropriate corporate or edgy marketing attire at a suitable location — a whiteboard perhaps, an AV projector.

Go ahead present your vision of the future in a succinct, and sizzly deck.

I will also have a nice PowerPoint presentation prepared.

20 minutes each. 20 minutes to blow people’s minds.

And then, the people can decide provided they are still awake.

Mr. Boudinot, author of The Littlest Hitler and soon to be released novel Egg and Sperm, I am calling you out. I challenge you to a PowerPoint-off. I demand this, or I demand your immediate concession to my generally sensible and cogent explanations and thoughts about the future of Richard Hugo House.

Name your time. Name you place. Check my Outlook calendar and schedule a rumble.

Thank You,

Matt Briggs

Comments are closed

Obama’s O on his Jet Plane

Obama 2008.jpg
This came across the tubes and cables of the Internet, “I don’t get this one—-it is offensive… WHAT A DISGRACE!!! AND HE IS ALL AMERICAN????” — Denise Emch. Emch is presumably offended by the fact that previous campaigns have used the logo of the American flag. Obama, though, is using his own sunrise/flag “O” logo. The transgressions here is somehow an affront to the fixed iconography of American principles.

Continue Reading →

Comments are closed

Star Wars + Wikipedia = Space Junk

Star Wars WikipediaWikipedia’s inaccuracies put forth by people interested in defaming famous figures is well documented. I still find the communal junk heap of information useful in the same way I find I find a casual search in Google useful: it will tell me what people are thinking and what people have felt strongly enough to post. I have long ago lost any sense that what I’m reading is actually factually accurate, but I do take a random webpage to have some insight into whatever subject I’m trying to read. When I want to find facts, though, I consult ProQuest and hope the newspaper article I pull has been fact checked.

Yesterday I found myself reading the voluminousness and apparently rapidly growing body of knowledge of Star Wars. And realized that Wikipedia easily has more information relating to George Lucas’s fantasy life than it does to the entire city of Seattle or the Pacific Northwest. I discovered that Jedi when using lightsabers use eight historical combat styles. Yoda and Darth Maul use Form IV – Ataru which means the Jedi uses the Force to throw around their body. Darth Maul is that face-tattooed guy from the first of the new movies played a marital artist who threw himself around. Yoda flitted around in the same series during sword fights, a random green CGI blob.

I’m enough of a geek that it didn’t occur to me that I was reading this on the web’s version of the encyclopedia. It didn’t occur until I began to the history of the Jedi space craft that these entries put to bed the entire idea of inaccuracies about famous people, the infiltration of the entire Wikipedia encyclopedia with the gnats of buzz marketers, that the entire foundation of a communal repository of fact is flawed since it assumes that fact has any kind of residence inside the communal mind.

Even if we were able to create a digital version of Borge’s Library of Babel, I suspect people would spend more of their time consulting this complete set of all human knowledge looking for information on Star Wars or Lost or finding crackpot theological scrolls. Well, maybe not everyone, but I would.

Comments are closed