Using Flask on dreamhost problems

We're updating our website here at Thought Into Design to reflect our firmer move into education. We wanted a site that discussed the kinds of things we're offering so that people know.

But a new design meant uploading a new site! And we had problems with Dreamhost and thought to document them here in case they're any use to anyone else.

Matt Carrier has a great page on setting flask up at Dreamhost that we used before for our Salstat page. All was going well until we had to install Flask itself with...

easy_install flask

...when our server told us that it could not be installed! Fundamentally, it could not find the flask package which seemed odd. We tried upgrading pip and setuptools but that crapped out too.

Searching for flask
Reading http://pypi.python.org/simple/flask/
Couldn't find index page for 'flask' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading http://pypi.python.org/simple/
No local packages or download links found for flask
error: Could not find suitable distribution for Requirement.parse('flask')
(test.thoughtintodesign.com)[clayton]$ cd ../public/
(test.thoughtintodesign.com)[clayton]$ easy_install flask

Our solution was to change directory to the lib directory, then download and install pip manually like this...

lib/python2.7/site-packages
wget bootstrap.pypa.io/get-pip.py
pip install flask

...and Flask installed nicely.

Research trip to Singapore

We've just returned from a long research trip where we attended the Speciality Fine Foods Asia event in the Suntec Convention Centre in Singapore. We learned a lot of great things about the food and drinks market in Singapore (and south-east Asia!) and are developing the markets in the region for user research and UX design.

We're also planning some workshops in south-east Asia where we'll be training people to undertake UX activities such as research, design and usability testing. If you're interested in participating, you can contact us on alan@thoughtintodesign.com, Twitter at @ThoughtN2Design, or Facebook.

International research & conferences

I've been invited to give the plenary talk at the PSITE conference in Tacloban, Philippines. It is a real honour to present an outline of user research and user experience design to such an important group of IT educators and I hope I can reward them with a fantastic talk.

The conference is 19–21 October 2017 and I'll be there with my company, Thought Into Design, the whole time.

We're also planning on running some UX workshops / training sessions the following week whilst we're in the Philippines. I can be contacted at alan@thoughtintodesign.com if you'd like to know more.

My partner and I just returned from Singapore and the Philippines after doing some research for DIT. It was a great few weeks and we gained some surprisingly useful insights for this client.

Statistics Routines

In addition to work on SigmaSwiftStatistics, we're writing some statistics libraries for other languages.

We're a bit sneaky, because we find this a great way to get familiar with languages old and new, but it also provides a useful service for everyone else.

Currently, we're writing some routines in Ada to complement the Swift ones. You can get the Ada code here. We'll be creating a Prolog repository soon.

Consulting at the Department for International Trade

Thought Into Design Ltd is heavily involved with some research activities at the Department for International Trade (DIT). We're very excited to be collaborating with DIT on a range of research projects. We've already undertaken some usability testing and we're hoping to add our skills and knowledge to all the rest that the DIT has available.

This is coming very soon after our stint at helping the Office for National Statistics improve their Interdepartmental Business Register (IDBR), which is a register of 2.1 million UK businesses and fully compliant with the European Union regulation on harmonisation of business registers for statistical purposes (EC No 177/2008).

Trying out Swift

Lately, at Thought Into Design, we've been trying out Swift. This was partially so that we could write apps for iOS and OSX (or is that MacOS now?) but it is also good to learn the foibles of a new language's syntax.

Our most recent work has myself (Alan) contributing towards an open source statistics library called SigmaSwiftStatistics. It's been great fun so far and we've contributed a few descriptive functions (to be cleaned up and made better by the project's maintainer, Evgenii Neumerzhitckii) such as the coefficient of variation, geometric mean, harmonic mean, skewness, kurtosis, Hyndman & Fans 9 quantile methods and a handful of nonparametric tests like a decent mode routine and a routine to rank data.

The mode is more useful than, say, that found in SPSS because it reports not just the modal value but all the indices at which it occurs. It's been some years since I used SPSS, but when I did, I recall it only provided the first index of occurrence. Sometimes, it's useful to know everywhere that it occurs.

Anyway, we've been committing to this project and are sure you'll find it useful. It's in Cocoapods so it should be simple to use into a project.

Over the last few evenings, we've been coding up a routine for univariate analysis of variance (both within and between subjects). It's passed its initial tests (using ANOVA tables generated from SPSS) so the code is almost ready to go. We hope to get it committed sometime this weekend.

We're also keen to produce some decent nonparametric tests. When people write routines, there is a tendency to focus on tests for parametric data and tests for nonparametric data are the forgotten children. We've found nonparametric tests to be excellent in many real-life circumstances (heavily skewed or kurtotic data, no normal distribution, scales of less than 11 items if I remember my Nunally correctly) so we want to ensure they are included.

In the future, I would like to investigate putting a Swift wrapper of some kind conforming to BLAS so that a massive range of linear algebraic routines (often heavily optimised) will be available. When using Swift, I find I miss Numpy and SciPy, and a layer on top of BLAS would help bring some of that serious power over.

Here's to statistics on Swift!

Fast prime numbers in Python

I spent some time recently on Project Euler and got side-tracked by the efficient calculation of prime numbers. After using a brute force method (iterating through a range of numbers and trying to find their factors), I read around and found a nice page at http://rebrained.com/?p=458, I found a good Stack Overflow page at http://stackoverflow.com/questions/2068372/fastest-way-to-list-all-primes-below-n. These inspired me to try again and I came up with the following routine. It's faster than most but not as fast as primes6 on the first page I linked to when generating more than approximately prime numbers up to 350,000-400,000. Below that, nothing seems to touch it.

It's also different. It uses numpy (which is cheating, in a way) but does the job well. I like it because it seems more understandable once you've grasped that the routine doesn't do any division. Instead, it's pure sieve operations on a vector of booleans. Anything found to be divisible by anything other than 1 and itself is marked as False, and the routine finishes by returning the indices of True values - which are primes.

I've triangulated the results by summing them and comparing the sums against those of other routines and there's no differences I've noticed yet.

import numpy as np

from math import sqrt

def ajs_primes3a(upto):

  mat = np.ones((upto), dtype=bool) # set up a long boolean array

  mat[0] = False # remove 0

  mat[1] = False # remove 1

  mat[4::2] = False # remove anything divisible by 2

  for idx in range(3, int(sqrt(upto))+1, 2): # remove anything else divisible

      mat[idx*2::idx] = False 

  return np.where(mat == True)[0] # return the indices which are the primes

I'm quite pleased with this early foray into optimising a routine but there's work to do compared to prime6. What I like is that it has no division and instead seems to be a pure sieve and doesn't create a long list of numbers.

I tried other versions with a half-series so that anything divisible by 2 just wasn't considered, but what I came up with just weren't as fast.

Times (msecs, same machine, best of 3-6 multiple runs)

             10k      100k     500k     1m       20m
prime6       0.001258 0.002722 0.007229 0.001229 0.22388
erat         0.005414 0.059047 0.333737 0.673749 15+ seconds
ajs_primes3a 0.000360 0.001897 0.008540 0.016952 0.70135

Up to 100k, mine leads but prime6 takes over strongly after that. Mine doesn't lose too much ground, considering, so it's best to think of mine as fast-ish but nicely understandable.