A few weeks ago, I started tinkering with our government’s HDB resale prices API . I had an idea to use recent transaction data to gauge if a listing on a property portal (e.g., property guru) is overpriced or fairly priced. Paralyzed by the sheer number of variables, I soon lost steam. Resuming after a two week hiatus, I scoped the coding down to just fetching recent transactions which matched a listing’s address (specific block and street), flat type (4 room or 5 room) and floor level (low, mid and high). Read more...

I extracted the JSON utility that I wrote previously into a standalone repository. Cleaned up the code and tests! Check it out here!

I set myself a challenge to implement an API endpoint in Common Lisp, in hope of putting to use a few things that I have learnt recently, defining structures creating a new project using cl-project:make-project adding and pulling in dependencies serving REST endpoints using hunchentoot web server library using rove testing framework to test drive implementation And so, I settled on coding up a mortgage calculator which takes in principal (loan amount), tenure (in years) and annual interest rate, and returns a list of monthly payments, each comprising of interest, principal repayment, and outstanding principal so far. Read more...

I extracted all the boilerplate to perform a REST get operation into rest-helpers: (defun get-json (url) (let ((json)) (ignore-errors (multiple-value-bind (data status) (drakma:http-request url) (if (= 200 status) (let* ((str (flexi-streams:octets-to-string data))) (setq json (json:decode-json-from-string str)))))) (unless json (format t "~&Unable to fetch from ~A~%" url)) json)) Using both rest-helpers and json-helpers (from previous post), to get a list of astronauts in space requires much less code: (defun get-astronauts () (mapcar #'format-astronaut (get-json-value `(:people) (get-json "http://api. Read more...

I wrote a helper get-json-value to extract values from json output produced by cl-json. (defun property-p (property) (funcall #'consp property)) (defun get-property-value (property) (funcall #'cdr property)) (defun get-property-key (property) (funcall #'car property)) (defun json-object-p (json) (and (listp json) (every #'consp json))) (defun json-array-p (json) (and (listp json) (every json-object-p json))) (defun json-p (json) (or (json-object-p json) (json-array-p json))) (defun get-json-value (keys json) (princ (list keys json)) (terpri) (cond ((null keys) json) ((json-p json) (let ((key (car keys))) (if (numberp key) (get-json-value (cdr keys) (nth key json)) (let ((property (find-if #'(lambda (property) (and (property-p property) (eq (get-property-key property) (car keys)))) json))) (if property (get-json-value (cdr keys) (get-property-value property))))))) (t nil))) For example, a json object produced by cl-json looks something like this: Read more...

This weekend, I learnt how to fetch and parse JSON using the drakma HTTP client and cl-json libraries. And some simple error handling as well. Tests: (ql:quickload :testlib) (use-package :testlib) (run (context "who is in space?" (context "people-p" (context "when key is people" (test "it should return true" (people-p (cons :people '())))) (context "when key is not people" (test "it should return false" (not (people-p (cons :foo '())))))) (context "get-name" (context "when name is present" (test "it should return name" (string= (get-name `(,(cons :name "foo"))) "foo"))) (context "when name is not present" (test "it should return nil" (eq nil (get-name `(,(cons :craft "bar"))))))) (context "get-craft" (context "when craft is present" (test "it should return craft" (string= (get-craft `(,(cons :craft "foo"))) "foo"))) (context "when craft is not present" (test "it should return nil" (eq nil (get-craft `(,(cons :name "bar"))))))) (context "format-astronaut" (test "it should format correctly" (string= (format-astronaut `(,(cons :name "foo") ,(cons :craft "iss")) nil) (format nil "~&foo (iss)~%")))) (context "get-people" (test "it should return people list from property pair" (equal (get-people (cons :people '(1 2 3))) '(1 2 3)))))) Implementation: Read more...

Writing a CSV parser is a good way to learn about streams and string formatting in Common Lisp. The goal is to take any CSV file and pretty print the contents. This is done by setting the width of each column to the widest value in that column + 1. Code for the CSV parser: (defun get-lines (input-stream) (let ((lines '())) (do ((line (read-line input-stream nil nil) (read-line input-stream nil nil))) ((null line) (nreverse lines)) (push line lines)))) (defun get-columns (line) (let ((columns '())) (do ((i 0 (+ j 1)) (j (position #\, line :start 0) (position #\, line :start (+ j 1)))) ((null j) (progn (push (subseq line i) columns) (nreverse columns))) (push (subseq line i j) columns)))) (defun get-rows (lines) (map 'list #'get-columns lines)) (defun get-fitting-column-width (rows n) (let ((get-nth-column-width #'(lambda (row) (length (nth n row))))) (+ (apply #'max (map 'list get-nth-column-width rows)) 1))) (defun get-fitting-column-widths (rows) (let ((cols (apply #'max (map 'list #'length rows)))) (if (zerop cols) '() (loop as n from 0 to (- cols 1) collect (get-fitting-column-width rows n))))) (defun format-rows (rows &optional (output-stream t)) (let* ((widths (get-fitting-column-widths rows)) (format-row (lambda (row) (let ((n 0)) (map 'list #'(lambda (col) (let ((w (nth n widths))) (incf n) (format nil (format nil "~~~AA" w) col))) row))))) (format output-stream "~{~&~{~A~}~%~}" (map 'list format-row rows)))) (defun format-csv (input-stream &optional (output-stream t)) (format-rows (get-rows (get-lines input-stream)) output-stream)) And the test output of tests written using the little test library I have been refining: Read more...

The previous version of test library had a number of problems, most importantly it relies heavily on the lexical variables indent and outcomes shared across multiple test runs. This meant that if test execution was terminated halfway and restarted for a set of tests, the indent and outcomes might not have been correctly reset, hence resulting wrong test output formatting. The fixed version: (defvar *test-output-stream* t) (defun make-spaces (n) (make-string n :initial-element #\space)) (defun format-outcome (depth name outcome &optional (output-stream *test-output-stream*)) (format output-stream "~&~ATest `~A` ~:[failed~;passed~]. Read more...

Putting my test library to use, I tried to implement a simple password strength checker. I immediately realised that I needed a better way to organize my tests and test output. The current test output looks like this: Test `it should return strong if it contains at least 1 lowercase alphabet, at least 1 uppercase alphabet, 1 special character, 1 digit and its length is at least 8` passed. Test `it should return weak if it contains no lowercase alphabet even though it has at least 1 uppercase alphabet, 1 special character, 1 digit and its length is at least 8` passed. Read more...

After learning a little more, I decided to apply what I have learnt to make our barebones testing library even more minimal. Before (LOC 14): (defmacro make-test (name predicate) `(lambda () (let ((p ,predicate)) (if p (format t "Test `~A` passed~%" ,name) (format t "Test `~A` failed~%" ,name)) p))) (defun run-tests (&rest tests) (let* ((score (lambda (p) (if p 1 0))) (run-test (lambda (test) (funcall test))) (scores (map 'list score (map 'list run-test tests))) (num-tests (length scores)) (num-passes (reduce #'+ scores))) (format t "~A out of ~A tests passed~%" num-passes num-tests))) After (LOC 12): Read more...