The need to deal with big data has stimulated the development of distributed systems such as Hadoop, Spark, than can cope with massive calculations. This opens the possibility of using agent-based models to run a big number of simulations and combine the outputs with bayesian estimations in order to produce models for estimating key statistical aggregates using easy to measure predictors. Let's assume, for example, that we want to develop a model to estimate the total production of a crop in an area using the number of people, animals, or cars transporting that crop on given selected roads during a given period of time. We first develop an agent-based model of the production and the transportation of the crop taking into account the existing roads network, the production points and the geographic positions of the main markets. Then we run the simulation with various parameters a sufficient number of times in order to have numerical estimations of the probability distribution of the number of observed people or cars on the key roads given the production. We can then use bayesian estimation to get the probability of the production being P if a number n of people or cars transporting the crop have been observed. Using multiple predictors may give good quality estimations that are more cost-effective than surveys or than can be used in combination with very light surveys to produce statistical information at lower cost.
Can such approach be the best way developing countries can leverage big data in order to produce better statistical information?
Note: It may be possible to use macroeconomic models such as computable general equilibrium models instead of agent-based models, provided we can model in a probabilistic way the individual behaviours that drive the transition from one global equilibrium to another global equilibrium (based for example on surveys about how individual behave in given situations: When do they decide that they have surplus to sell? Do they just go to the nearest market or their try different markets? Which transportation means they use, etc.)