Csv Parser
Writing a CSV parser is a good way to learn about streams and string formatting in Common Lisp. The goal is to take any CSV file and pretty print the contents. This is done by setting the width of each column to the widest value in that column + 1.
Code for the CSV parser:
(defun get-lines (input-stream)
(let ((lines '()))
(do ((line (read-line input-stream nil nil)
(read-line input-stream nil nil)))
((null line) (nreverse lines))
(push line lines))))
(defun get-columns (line)
(let ((columns '()))
(do ((i 0 (+ j 1))
(j (position #\, line :start 0) (position #\, line :start (+ j 1))))
((null j) (progn (push (subseq line i) columns) (nreverse columns)))
(push (subseq line i j) columns))))
(defun get-rows (lines)
(map 'list #'get-columns lines))
(defun get-fitting-column-width (rows n)
(let ((get-nth-column-width #'(lambda (row) (length (nth n row)))))
(+ (apply #'max (map 'list get-nth-column-width rows)) 1)))
(defun get-fitting-column-widths (rows)
(let ((cols (apply #'max (map 'list #'length rows))))
(if (zerop cols)
'()
(loop as n from 0 to (- cols 1) collect (get-fitting-column-width rows n)))))
(defun format-rows (rows &optional (output-stream t))
(let* ((widths (get-fitting-column-widths rows))
(format-row
(lambda (row)
(let ((n 0))
(map 'list
#'(lambda (col)
(let ((w (nth n widths)))
(incf n)
(format nil (format nil "~~~AA" w) col))) row)))))
(format output-stream "~{~&~{~A~}~%~}" (map 'list format-row rows))))
(defun format-csv (input-stream &optional (output-stream t))
(format-rows (get-rows (get-lines input-stream)) output-stream))
And the test output of tests written using the little test library I have been refining:
csv parser
get-lines
Test `it should return the correct lines from input stream` passed.
get-columns
Test `it should return the correct words` passed.
get-rows
Test `it should return the correct rows with columns in each row` passed.
get-fitting-column-width
Test `it should return the correct column width` passed.
get-fitting-column-widths
Test `it should return the correct column widths` passed.
format-rows
Test `it should format ouput correctly` passed.
format-csv
Test `it should read input stream correctly and output formatted csv` passed.
7 out of 7 tests passed
And finally running the parser on a number of CSV files from here.
Monthly transatlantic airtravel, in thousands of passengers, for 1958-1960:
(with-open-file (is "airtravel.csv" :direction :input) (format-csv is))
"Month" "1958" "1959" "1960"
"JAN" 340 360 417
"FEB" 318 342 391
"MAR" 362 406 419
"APR" 348 396 461
"MAY" 363 420 472
"JUN" 435 472 535
"JUL" 491 548 622
"AUG" 505 559 606
"SEP" 404 463 508
"OCT" 359 407 461
"NOV" 310 362 390
"DEC" 337 405 432
Average nightly viewship for 6 TV news magazines for 2009-2011:
(with-open-file (is "news_decline.csv" :direction :input) (format-csv is))
"Show" "2009" "2010" "2011"
"60 Minutes" 7.6 7.4 7.3
"48 Hours Mystery" 4.1 3.9 3.6
"20/20" 4.1 3.7 3.3
"Nightline" 2.7 2.6 2.7
"Dateline Friday" 4.1 4.1 3.9
"Dateline Sunday" 3.5 3.2 3.1
Rotten Tomato ratings of movies with Robert De Niro:
(with-open-file (is "deniro.csv" :direction :input) (format-csv is))
"Year" "Score" "Title"
1968 86 "Greetings"
1970 17 "Bloody Mama"
1970 73 "Hi Mom!"
1971 40 "Born to Win"
1973 98 "Mean Streets"
1973 88 "Bang the Drum Slowly"
1974 97 "The Godfather Part II"
1976 41 "The Last Tycoon"
1976 99 "Taxi Driver"
1977 47 "1900"
1977 67 "New York New York"
1978 93 "The Deer Hunter"
1980 97 "Raging Bull"
1981 75 "True Confessions"
1983 90 "The King of Comedy"
1984 89 "Once Upon a Time in America"
1984 60 "Falling in Love"
1985 98 "Brazil"
1986 65 "The Mission"
1987 100 "Dear America: Letters Home From Vietnam"
1987 80 "The Untouchables"
1987 78 "Angel Heart"
1988 96 "Midnight Run"
1989 64 "Jacknife"
1989 47 "We're No Angels"
1990 88 "Awakenings"
1990 29 "Stanley & Iris"
1990 96 "Goodfellas"
1991 76 "Cape Fear"
1991 69 "Mistress"
1991 65 "Guilty by Suspicion"
1991 71 "Backdraft"
1992 87 "Thunderheart"
1992 67 "Night and the City"
1993 75 "This Boy's Life"
1993 78 "Mad Dog and Glory"
1993 96 "A Bronx Tale"
1994 39 "Mary Shelley's Frankenstein"
1995 80 "Casino"
1995 86 "Heat"
1996 74 "Sleepers"
1996 38 "The Fan"
1996 80 "Marvin's Room"
1997 85 "Wag the Dog"
1997 87 "Jackie Brown"
1997 72 "Cop Land"
1998 68 "Ronin"
1998 38 "Great Expectations"
1999 69 "Analyze This"
1999 43 "Flawless"
2000 43 "The Adventures of Rocky & Bullwinkle"
2000 84 "Meet the Parents"
2000 41 "Men of Honor"
2001 73 "The Score"
2001 33 "15 Minutes"
2002 48 "City by the Sea"
2002 27 "Analyze That"
2003 4 "Godsend"
2004 35 "Shark Tale"
2004 38 "Meet the Fockers"
2005 4 "The Bridge of San Luis Rey"
2005 46 "Rent"
2005 13 "Hide and Seek"
2006 54 "The Good Shepherd"
2007 21 "Arthur and the Invisibles"
2007 76 "Captain Shakespeare"
2008 19 "Righteous Kill"
2008 51 "What Just Happened?"
2009 46 "Everybody's Fine"
2010 72 "Machete"
2010 10 "Little Fockers"
2010 50 "Stone"
2011 25 "Killer Elite"
2011 7 "New Year's Eve"
2011 70 "Limitless"
2012 92 "Silver Linings Playbook"
2012 51 "Being Flynn"
2012 29 "Red Lights"
2013 46 "Last Vegas"
2013 7 "The Big Wedding"
2013 29 "Grudge Match"
2013 11 "Killing Season"
2014 9 "The Bag Man"
2015 60 "Joy"
2015 26 "Heist"
2015 61 "The Intern"
2016 11 "Dirty Grandpa"