I am a Professor at the Santa Fe Institute. I work on problems at the interface of physics, computer science, and mathematics, such as phase transitions in statistical inference. When the amount of noise in a data set crosses a critical threshold, it can suddenly become impossible to find underlying patterns in it, or even tell if a pattern is really there. This includes finding communities in social and biological networks, or clusters in high-dimensional data, or structure in noisy matrices and tensors. How can we locate these phase transitions, and what informational and computational barriers do they create?
I am also working on issues of transparency and accountability in algorithmic and human decision making, such as risk assessment for pretrial defendants, and figuring out what kinds of transparency and auditability algorithms need to have in order to tell whether antidiscrimination laws such as the Fair Housing Act are being violated. I am especially interested in demystifying issues in machine learning and statistics so that a broader public can make critical decisions about whether and how algorithms should be deployed, and in turn making sure that my own work is grounded in the actual needs of policymakers and advocates.
I have also worked on phase transitions in search and optimization problems, where problems suddenly become unsolvable when they become too constrained; quantum computation and quantum algorithms for the Graph Isomorphism problem; the computational complexity of predicting physical systems, and of solving systems of equations; percolation, topological defects, and Monte Carlo algorithms; games, tilings, and cellular automata; the stability of financial markets, epidemics in networks, and universality in human language; the combinatorics of proof-of-stake blockchains; and braided orbits in the three-body problem.
Criminal Justice: Here is a study of pretrial detention, specifically presumptions of dangerousness that would recommend detention for certain classes of felony defendants. My coathors at UNM's Institute for Social Research and I found that many legislative proposals for these presumptions are highly inaccurate; the broadest proposals are little more accurate than detaining a random sample of defendants.
In another study we take a closer look at pretrial rearrest. We find that across many categories of felony defendants, including those charged with serious felonies, rearrest during the pretrial period for high-level felonies is very rare: 1% for 2nd degree and about 0.1% for 1st degree. The most common type of rearrest is for 4th degree felonies, and about a third of rearrests (even those classified as violent) are for misdemeanors or petty misdemeanors. Based on this we urge judges and other stakeholders, and validation studies of risk assessment algorithms, to go beyond measures of recidivism or "New Criminal Activity" that lump crimes of multiple severities together.
Housing: in 2019 we submitted a comment on proposed changes to Housing and Urban Development regulations that would allow discrimination lawsuits to be deflected by algorithms for lending and risk assessment of buyers and tenants, without adequate transparency or auditing for fairness. I also presented these ideas to the staff of the Financial Services Committee of the U.S. House of Representatives.
Recently read books
Here are two animated applets by UNM undergraduate Rory McGuire: Union-find (with path compression), and 2-3-4 Trees.
I used to want Martin Gardner's old job, but I think Vi Hart would be even better at it.
One of the highest professional honors I have received.
For many years, I was blessed with a cat named Spootie.
I am a big fan of Vladimir Nabokov. Here are some of his favorite words.
Here are definitions of words from the Alchymist's Journal by Evan S. Connell (recommended to me by the inimitable Cosma Shalizi).
Here are a few poems by my grandfather, Louis Untermeyer.
I have been known to cite fictional books.
I and Mats Nordahl are the editors-in-chief of the Journal of Unpublished Results, and I also edit the Journal of Weird-Ass Shit.
Finally, here is a list of restaurant reviews for Santa Fe and Paris. Of course, these are my own personal opinions, which, though correct, may or may not be shared by my employers. They are also often sadly out of date.