Bash one liner to randomize lines in file
October 18, 2007 at 11:57 AM | categories: unix, system administration, linux | View Comments Discovered that the bash shell has a variable called $RANDOM, which outputs a pseudo-random number every time you call it. Sweet! Allowed me to randomize the lines in a file for a process I needed to do, thusly:for i in `cat unusual.txt`; do echo "$RANDOM $i"; done | sort | sed -r 's/^[0-9]+ //' > randorder.txtIn other words, put a random number on every line, sort the file, then take off the random numbers. Worked like a charm.
iPhone fun
September 27, 2007 at 12:22 AM | categories: python, unix, apple, iphone | View Comments # uname -a Darwin Matt Michie's iPhone 9.0.0d1 Darwin Kernel Version 9.0.0d1: Fri Jun 22 00:38:56 PDT 2007; root:xnu-933.0.1.178.obj~1/RELEASE_ARM_S5L8900XRB iPhone1,1 Darwin # python Python 2.5.1 (r251:54863, Jul 27 2007, 12:05:57) [GCC 4.0.1 LLVM (Apple Computer, Inc. build 2.0)] on darwin Type "help", "copyright", "credits" or "license" for more information. Mmmmm tasty! iPhone with SSH, Python, BSD subsystem. Life is good.Regular Expressions, Lisp, SQL, Parsing, Domain Specific Languages
June 10, 2007 at 03:34 PM | categories: code, software engineering, lisp, philosophy, programming, unix | View Comments I've been trying to code some more on Project Shelob (my web server) in my spare time. I'm to the point of needing a configuration file, so I can start up the server using different ports and directories for testing. Speaking of testing, I'm also to the point of needing automated test suites. I was refactoring some of the HTTP code, and when I got done, it was far more readable, and there was much rejoicing! Unfortunately, two days later I discovered I had introduced a subtle bug in keep-alive handling during a 404 event. Oops. Anyway, I decided to use JSON as my configuration language. Simple, accommodated everything I needed, and later I would be able to easily write an AJAX GUI front end to configure the whole thing. Should be slick, right? Not as easy as it might sound. Though I have written parsers by hand, I'd rather not. Ok, so I'm using C++, surely someone has written an easy to use open source library that I can just stick in my rules and get out a nice data structure, right? Well, kind of. There is Boost Spirit which would do everything that I want it to do, but it also required me translating the EBNF grammar of JSON into Boost's strange amalgamation of YACC and C++. Okay well and good, but surely there is something better? After some more searching, I run across ANTLR which seems to be the spiritual successor to LEX and YACC/Bison. It even has a nice Java GUI and someone had kindly done the ANTLR rules for JSON. Check out the graphical goodness:
"One of the most obviously DSLy parts of the world is the Unix tradition of writing little languages. These are external DSL systems, that typically use Unix's built in tools to help with translation. While at university I played a little with lex and yacc - similar tools are a regular part of the Unix tool-chain. These tools make it easy to write parsers and generate code (often in C) for little languages. Awk is a good example of this kind of mini-language."While I've been using SQL, regular expressions, awk, lex, and yacc for years, I'd never really classified them in my mind as DSLs. I've been well aware of the power of small specialized utilities aggregated together to perform a bigger task and why UNIX has been so successful at this, but I hadn't made the leap to apply this to my programming. Fowler continues:
"Lisp is probably the strongest example of expressing DSLs directly in the language itself.. Symbolic processing is embedded into the name as well as practice of lispers. Doing this is helped by the facilities of lisp - minimalist syntax, closures, and macros present a heady cocktail of DSL tooling. Paul Graham writes a lot about this style of development. Smalltalk also has a strong tradition of this style of development."I've heard "grey-beards" and academics talk about the power of Lisp for years, and though I did some trivial functional programming in college, I've dismissed the rants of the Lisp guys as nothing more than rants. Today though, the ideas are crystallizing in my head, and I'm excited to explore this more.
Microsoft, true innovation
June 03, 2007 at 10:54 AM | categories: unix, philosophy, humor, microsoft | View Comments Wes: check out introducing pipes Matt: "Those who do not understand Unix are condemned to reinvent it, poorly." Matt: I hear vista finally has symlinks. Wake me up when they invent mount points and finally kill drive letters Wes: I think you can do that somehow. Matt: yeah sure, and break everything *nerd rage* Wes: yeah, junction point. junction points (technet) Matt: "Those who do not understand Unix are condemned to reinvent it, poorly." Update, Wes says, if you want to know more see his blogs at:Amazon S3 Backup Solution
May 21, 2007 at 12:48 PM | categories: unix, web 2.0, system administration, perl | View CommentsAlthough I've had an Amazon Simple Storage Service account for awhile, I haven't used it. For those of you who aren't familiar with S3, Amazon has opened up their resources for everyday people to use. In this instance, you can use their servers as a place to dump your files online. Currently they charge $0.15 per gigabyte of storage used as well as a fee for the bandwidth to transfer it back and forth.
With this setup, they take care of the administration, backup, redundancy, troubleshooting, and the storage scales to whatever you need automatically. I've been searching for a good backup script so I can backup all the stuff I have running on this web-host, but most of them have been beta to this point or a pain to setup. Today I finally installed Brackup through CPAN, along with all the requisite Perl modules. I've already tested a backup and restore and it seems it will fit my needs well.
Apache Logs Compress Well
February 13, 2007 at 01:06 PM | categories: apache, unix, http | View Comments1 342 625 990 / 105 314 712 = 12.748703Yes, log files contain a lot of redundant information. I enjoy seeing over 12X compression on a file!
Simple unix tools in Haskell
September 22, 2006 at 10:58 AM | categories: unix, programming | View CommentsSimple unix tools written in Haskell.
This is intended as a beginners tutorial for learning Haskell from a "Lets just solve things already!" point of view. The examples should help give a flavour of the beauty and expressiveness of Haskell programming.
Next Page ยป