The Azimuth Project
Experiments in fitting bivariate power laws

Experiments in fitting bivariate power laws


This page looks at fitting a power law to an urban statistics dataset. The freely available dataset found is the relationship between city size and the number of list of cities with the most high-rise buildings. A high-rise is defined as a structure at least 35 meters or 12 stories tall. It’s unclear that this should be determined by a power law, but given tall buildings are generally erected to accomodate high-level economic activity, which is the kind of thing that is argued to depend on population size via a power law. The data comes from Wikipedia and no attempt has been made to verify the data or apply any “corrections” such as removing outliers. The data is also right censored at a minmum of 100 high-rise buidlings.

Since we’re considering a situation where the dominant deviation from a power law is likely to, be due to e.g., a city being underdeveloped or overdeveloped due to unique factors, rather than measurement error, we’ll refer to differences from the basic power law as “deviations” rather than “errors”.

The aim of this page is not to find “the correct” power law fit for this data, but to look at the question of whether convincing power law fitting is possible in the absence of at least an idea of what kind of deviations are believed to be occurring.

Visualising the dataset

power law test original data

Original data: xx-axis is vv, yy-axis is bb.

power law test double log

Double logarithm data: xx-axis is logv\log v, yy-axis is logb\log b.

Arguably both these plots could suggest there are some outliers which may for some reason be generated by an anomalous process. Many fitting methods, including maximum likelihood can be heavily skewed by such outliers and they should be removed for best results. However, removal can only be done in a principled way if there are pre-existing theory justifying their exclusion, and we’re considering the case of trying to fit a power law in advance of building a theory.

In order to see the visual effect of binning in the double logarithm domain, here is a plot using 20 equispaced bins. Note how this makes the data look like a line fit would bemuch more reasonable than the unbinned data.

power law test double log

Since we’re fitting using models in the original data rather than the log variables we don’t do anything more with this reduced data set.

Fitting using likelihood given a devation model

We look at two very simple deviation models: additive normal deviation in the original data and additive normal deviation in the original data (which is log-normal deviation in the double logarithm data). Here vv is the city population and bb is the number of high-rise buildings.

modelhenceelt log likelihoodargminC @ min
b~Cv β+N(0,σ)b ~ C v^\beta+N(0,\sigma)bCv β~N(0,σ)b-C v^\beta ~ N(0,\sigma)σ 2 i(b iCv i β) 2\sigma^{-2}\sum_i (b_i-C v_i^\beta)^2[0.6925,0.693][0.6925,0.693]0.03260.0326
b~Cv β(1+N(0,σ))b ~ C v^\beta(1+N(0,\sigma))b/(Cv β)1~N(0,σ)b/(C v^\beta)-1 ~ N(0,\sigma)σ 2 i(b i/(Cv i β)1) 2\sigma^{-2}\sum_i (b_i/(C v_i^\beta)-1)^2[0.653,0.6535][0.653,0.6535]0.09130.0913

(Since this deviation measures the “vertical” distance to the fitted curve, this model is essentially assuming that cities produce more or fewer hirise buildings for their size: a deviation based on distance to the closest point on the curve would embody different assumptions.) For comparision, least squares fitting in the double logarithm domain is equivalent ot an original data model of

b~Cv βexpN(0,σ)b ~ C v^\beta \exp N(0,\sigma)

The model technically has three parameters (C,β,σ)(C,\beta,\sigma). However, providing σ\sigma is essentially independent of the other parameters and the vv values it only scales the likelihood and so does not affect the location of the minimum. Additionally, using the analytical differentiation of the netative log likelihood the value of CC at the minimum has a simple form in terms of the data and β\beta. So in these cases maximum likelihood parameter fitting collapses to just a 1-D search over β\beta.

Plots of the negated log likelihoods of the two models (over the same range of β\beta values). Note that because the deviation term has a different meaning in the different models, the fact the two functions have very different ranges of values doesn’t mean anything.

power law test model 1 NLL

Model 1 negative log likelihood plot: xx-axis is β\beta, yy-axis proportional to negative log likelihood.

power law test model 2 NLL

Model 2 negative log likelihood plot: xx-axis is β\beta, yy-axis proportional to negative log likelihood.

The β\beta range within which the minimum falls, and approximately the corresponding CC value, are recorded in the table. As can be seen there is a dramatic difference between the two values.

power law test original data + overlay

Original data with two fitted power laws overlaid.

power law test double log

Double logarithm data with two fitted power laws overlaid.

It’s highly debatable if either of these fitted power laws are a good fit to the data, but it is clear that very different results are obtained depending on the deviation model chosen. This raises doubts whether it’s possible to find power laws in data with medium to high deviation purely by fitting procedures.

Source scripts will be available here.