Topic: unix

Image by dannyman via Flickr

One of my favorite tools is "grep." That gives away the fact that I spend more time on the command line than many. One of the things I originally loved about OS X was that I could fire up a terminal and use the machine just like Unix (yeah, Linux was a new fangled thing for me).

Recently I complained about always having to look up a certain switch for grep and Weldon Dodd tweeted "if you write a blog post about grep, maybe others will commit the switch to memory too, and when I say others, I mean me." So, here it is.

Grep is an acronym from "global, regular expression, print" three commands you often did in a row inside ed, a primitive Unix editor (vi is slightly better, emacs is the best, of course). As an aside, knowing some rudimentary ed commands is a good thing for any sysadmin because it's always available, even in single-user mode.

Grep is used to searching files. I was using it this morning to search for strings in old email mailboxes to find a Quicktime registration. The fact that grep accepts regular expressions makes it a very powerful tool for finding data in files.

There are only a few flags I commonly use:

• -H - print the filename of any matching files. If you grep a file glob and get match you want to know which file in the glob the match was in
• -P - use Perl-style regular expressions

There are others, just grep --help to see them.

One thing you will commonly want to do is grep through directories. Grep allows you to recurse, but I am in the habit of using it within find to do the same thing:

find . -exec grep -H "Quicktime" {} \;

Why do this instead of using the built-in recursive features? simply because I know find well and when I want it's powerful filtering features to work with, this makes them readily available. For example, if knew I wanted to search for "Quicktime" in files that were created in my Documents folder within the last week:

find ~/Documents -ctime 7 -exec grep -H "Quicktime" {} \;

So, why not use Spotlight? I do. But sometimes I want a scaple rather than a chainsaw and grep combined with find give me that.

Tags: unix osx

Image via Wikipedia

Jordan Hubbard, Apple's Director of Engineering of Unix Technologies, spoke at LISA '08 last week. Most people are commenting on the date he gave for the release of Snow Leopard (10.6), the newest version of OS X. I have to admit, I'm ready for some stability improvements, but I was much more intrigued by the details of his talk (PDF).

He spent the bulk of his talk on technical features in Leopard (10.5) that many aren't aware of. He starts with a number of security improvements in Leopard: file quarantine, sandbox, package and code signing, application firewall, parental controls, non-executable (NX) data, address space layout, and randomization. I was completely unaware of most of these improvements.

Jordan also talks about the Unix improvements in Leopard. Leopard is fully Unix compliant. But more than that includes a number of additions like DTrace, Launchd (complete), ASL (replacement for syslog), a read-only version of ZFS (for future compatibility) with a read/write version available. He also talked about Apple's evolving open source strategy.

Last, he talks about improvements coming in OS X that will help developers take better advantage of the multicore chips and sophisticated GPUs that already ship with most Macs. Future kernels will provide better facilities, along with APIs, for managing multi-threaded programs. He says:

Forget everything you thought you knew about multi-threaded programming (and, as it turns out, most developers didn't know much anyway). The kernel is the only one who really knows the right mix of cores and power states to use at any given time - this can't be a pure app-driven decision

I don't know if there's audio or video of the talk available, but it would be very good to hear firsthand.

BTW, anyone know what "LWFLAF" stands for? Jordan uses it as some kind of metric in discussion the various versions of OS X, but I couldn't figure out what it meant.

Tags: osx apple unix security

One of my grad students just went to remove some unwanted, automatically created files in his directory and accidentally deleted some things he wanted. I use a script to do clean ups to prevent these kinds of silly errors (which we're all prone to). Here's the script:

#!/bin/bash

if [ ! -e $HOME/.rmd ]
then mkdir $HOME/.rmd
fi

find $HOME  \( -name '.rmd' -prune \) -o \
  \( -name '*~' \
     -o -name ',*' \
     -o -name '#*#' \
     -o -name '*.bak'\
     -o -name '*.backup' -atime +5\
     -o -name 'core'\
  \) \
  -print -exec mv -f {} $HOME/.rmd \;

find $HOME/.rmd -atime +5 -exec rm -f {} \;

The script creates a directory called .rmd if it doesn't exist, finds files matching a certain set of patterns to that directory, and finally removes things in that directory that were moved there more than five days ago. It's not perfect--files with the same name are just moved over the top of each other.

I name it "clean" and put it in my personal bin directory. You might add or delete individual line items depending on what kinds of files your programs create. When I was a grad student, disks were expensive, and worked on a system that enforced quotas, I ran it in a cronjob once a day. Now I just run it whenever things look ugly--the same approach I have to dusting.

Building or modifying a script like this can be dangerous since a bug could cause things you care about to be systematically removed. I recommend testing it on an account that doesn't have anything you care about in it before you blindly trust it.

One last thing: I used Linux in the title, but this will obviously work in anything with bash and find including varieties of Unix and OS X. These days I'm running it on OS X rather than Ultrix or 4.3BSD. Not all versions of find have a "prune" option.

Tags: unix linux sysadmin programming osx

Steve Yegge rants, in reference to software design, that crap is still crap, no matter how many rubies you swallowed. If software design interests you, then you'll enjoy this--even if you don't agree.

As I was reading this, I was reminded several times about Scott Rosenberg's article on Charles Simonyi, Anything You Can Do, I Can Do Meta. Simonyi, who was the force behind Office at Microsoft and arguably the richest programmer in the world, is hot on the heels of a programming methodology he calls "intentional programming" and has a company to develop it Intentional Software.

The basic point is described in the article by means of a fable. The bottom line: don't build systems, rather build systems that build systems. Steve's not saying the same thing exactly, but there's a similarity of purpose, if not execution.

Tags: software programming design unix emacs