SmartCam

A scalable cloud-based video monitoring system (with motion detection, face counting, and image recognition) using Raspberry Pis. Most of the processing is done in the cloud, so the computing power of the Raspberry Pi is not a limitation.

Website
Backend Code
Frontend Code

Close

Shortest Path and PageRank algorithms applied to the Wikipedia graph dataset

Two of the assignments of the course, using with a graph dataset of almost half a million nodes.

Shortest Path
PageRank

Close

Forest Cover Type Prediction

Kaggle competition: predictions of the predominant kind of tree cover in four wilderness areas located in the Roosevelt National Forest of northern Colorado, from strictly cartographic variables such as elevation and soil type.

IPython Notebook

Close

Redefining the job search process

A pipeline that combines data from Indeed API and the U.S. Census Bureau to select the best locations for data scientists based on the number of job postings, housing cost, etc.

GitHub
Paper
Website

Close

A fresh perspective on Citi Bike

An interactive website to visualize NYC Citi Bike bicycle sharing service.

Website
GitHub

Close

Investigating the Effect of Competition on the Ability to Solve Arithmetic Problems

A randomized controlled experiment in which 300+ participants were assigned to a control group or one of two test groups to evaluate the effect of competition (being compared to no one or someone better or worse).

Paper
GitHub

Close

Experimental design to measure the impact of online advertising on the sales of a car manufacturer’s dealer network

A project for Google (working at Conento) to measure the effect of YouTube ads. Responsible for the whole project (excluding the econometric model to estimate the increase in advertising spend in the test group): designed a matched-pair, cluster-randomized experiment, which involved selecting the test and control groups from a sample of 50+ cities in Spain based on their sales-wise similarity over time.

Paper
Slides (Spanish)
GitHub

Close

Demonstration

We can fit the population of laptops to a Bernoulli distribution with probability \(p\) of being a Mac rather than a PC. Hence, in a sample of \(N\) laptops, the number of Macs would be \(N_1 = p \cdot N\), and the sample mean and variance are, respectively:$$\begin{aligned}\bar{x} &= \frac{1}{N}\sum_{i=1}^N x_i = \frac{N_1}{N} \\ &= p \\ s_x &= \frac{1}{N-1}\sum_{i=1}^N \left(x_i - \bar{x}\right)^2 = \frac{N_1(N-N_1)}{N(N-1)} \\ &= \frac{N}{N-1} p (1-p)\end{aligned}$$Our hypothesis is that Macs are more common than PCs:$$\begin{aligned}H_0&: \bar{x} = 0.5 \\ H_1&: \bar{x} > 0.5\end{aligned}$$To be able to reject the null hypothesis at the \(\alpha\) confidence level, the following inequality (where \(t = t_{1-\alpha, N-1}\)) should hold:$$\begin{aligned} \frac{\bar{x} - 0.5}{s_x/\sqrt{N}} &\geq t \\ \frac{p-\frac{1}{2}}{\sqrt{\frac{p(1-p)}{N-1}}} &\geq t \\ \frac{2p-1}{2} &\geq t\sqrt{\frac{p(1-p)}{N-1}} \\ (2p-1)^2(N-1) &\geq 4t^2p(1-p) \end{aligned}$$$$4p^2(N+t^2-1)-4p(N+t^2-1)+(N-1) \geq 0$$For simplicity, let's denote \(N+t^2-1\) by \(\theta\):$$4\theta p^2-4\theta p + N-1 \geq 0$$$$\begin{aligned}p &\geq \frac{4\theta+\sqrt{16\theta^2-16\theta(N-1)}}{8\theta} \\ &= \frac{1+\sqrt{1-\frac{N-1}{\theta}}}{2} \\ &= \frac{1}{2}\left(1+\sqrt{\frac{\theta-N-1}{\theta}}\right) \\ &= \frac{1}{2}\left(1+\frac{t}{\sqrt{\theta}}\right)\end{aligned}$$Hence the percentage of Macs must exceed \(50\%\) by \(50 t/\sqrt{\theta}\) percentage points to be significantly higher than that value (at the \(\alpha\) level).
As \(N\) increases, \(t_{1-\alpha, N-1}\) approaches \(\Phi^{-1}(1-\alpha)\) and \(\theta\) approaches \(N\); since \(N_1\) must be an integer (the smallest one that is greater than \(N \cdot p\)), the approximation$$\begin{aligned} N_1 &\simeq \left \lceil \frac{N}{2} \cdot \left(1 + \frac{\Phi^{-1}(1-\alpha)}{\sqrt{N}}\right) \right \rceil \\ &= \left \lceil \frac{N}{2} + \frac{\Phi^{-1}(1-\alpha)}{2} \cdot \sqrt{N} \right \rceil \end{aligned}$$works even for small values of \(N\).

Close

Prediction of customer churn for a mobile network carrier

Predictions from a sample of 45,000+ customers.

Paper (Spanish)

Close

Different models of Harmonized Index of Consumer Prices (HICP) in Spain

Forecasts based on exponential smoothing, ARIMA, and transfer function (using petrol price as independent variable) models.

Paper (Spanish)

Close

Data Scientist

Juanjo's main strengths were his ability to solve complex problems by developing innovative ideas, combined with his smart focus on time and resources to execute them. He worked hard to make sure results were achieved, adapting very well to our work environment and building an excellent relationship with his mates. He is intelligent, positive, and rigorous, and a very generous person that is just a pleasure to work with.

Close

Master of Information and Data Science

Juanjo was an excellent student in my course on experiments and causal inference. He asked terrific questions, turned in great problem sets, and did a very conscientious job on a team project experimenting with how perceptions of competition affect productivity in math problems.

Close

Master of Information and Data Science

Juanjo was my student at UC Berkeley in a master's-level class on the design and analysis of randomized experiments for corporate research. He was without a doubt one of the stand-out students. I would highly recommend his skills. His performance was excellent.

Close

Master of Information and Data Science

I was impressed by the presentation Juanjo made in my class on research design and applications for data analysis. Juanjo demonstrated great storytelling skills, giving a convincing demonstration of the need for his proposal. He can articulate a good research question – simple, direct, specific, and yet still challenging – and can handle tough questions from an audience. He has a strong grasp of research design, with really great description of why he recommended the adoption of an experimental design and good discussion of the method itself.

Close

Master of Information and Data Science

Juanjo is that wonderful combination of capable, smart, hard-working guy who also happens to be highly personable. I'm happy to recommend him.

Close

M.S. in Statistical and Computational Information Processing

Juanjo was an excellent student in my Master's course "Neural Networks and Statistical Learning" at the Technical University of Madrid. He showed a very high motivation and capability not only to grasp fundamental aspects but also to address real problems. In addition, Juanjo is a very open a communicative person which makes him easy to share ideas and organize tasks.

Close

Sales Director, T&M

I had the opportunity to work with Juan Jose and it was really easy to choose and run the installation. Not a lot of people understand the difficulties to choose an appropriated equipment to do a specialized job. Juan Jose does! Congratulations!

Close

Sales Director, T&M

Juan Jose is easy to work with, has a strategic mind, a genuine interest in people and other cultures. He easily understands complex matters and is a real team player.

Close

Seattle Rain

You’re about to get on a plane to Seattle. You want to know if you should bring an umbrella. You call 3 random friends of yours who live there and ask each independently if it’s raining. Each of your friends has a 2/3 chance of telling you the truth and a 1/3 chance of messing with you by lying. All 3 friends tell you that “Yes” it is raining. What is the probability that it’s actually raining in Seattle?

  Facebook interview question posted on Glassdoor.

Most answers posted on the link above are:

which is the probability that all friends say it is raining provided that it’s true… but we are interested in the opposite, the probability that it is raining provided that all friends say that.

Another common answer is based on a correct idea:

where

$h$ is a binary event, so:

But those answers then mix joint and conditional probabilities, getting yet a wrong result (which is correct only if the probability of rain in Seattle is $0.5$):

$\left(2/3\right)^3 = 8/27$ is not the joint probability that all friends say the same thing (it is raining) and that is true, but the probability that all friends say that it is raining conditional on the fact that it is raining.

Let’s consider the two extreme cases to confirm that the solution above is not always correct. If it is always raining in Seattle,

If it never rains in Seattle,

The tricky part of the problem, since we may be biased to think that the solution is fixed, is that there is an unknown parameter ($p$, the probability of rain in Seattle), which the solution depends on. We need to apply the Bayes’ theorem, which states that the posterior probability (the probability of an hypothesis $H$ conditional on a given body of data (the evidence, $E$) can be calculated as:

Let $q$ the probability that one friend tells the truth. Then, if there are $N$ friends:

We are told that $3$ is the number of friends and that $q = 2/3$. Hence:

Let’s see what the posterior probability would be for a few possible values of the prior probability:

Also, note that the posterior reaches a value of $1/2$ for a prior as low as $1/9 \approx 0.11$.

Let’s confirm that the solution above is right, simulating the problem in R (for all possibles values of $p$ from $0$ to $1$, in steps of $0.05$):

library(dplyr)
set.seed(12345)
# Simulate N cases for each probability of rain in Seattle (p)
# in the whole [0, 1] range, in steps of 0.05
N <- 5e3 # simulations (per value of p)
q <- 2/3 # prob of a friend telling the truth
data <- data.frame(p = rep(seq(from = 0, to = 1, by = 0.05), each = N))
data <- data %>% rowwise() %>% mutate(r = as.logical(rbinom(1, 1, p))) %>% 
  mutate(A = as.logical(ifelse(r == 1, rbinom(1, 1, q), rbinom(1, 1, 1 - q))), 
         B = as.logical(ifelse(r == 1, rbinom(1, 1, q), rbinom(1, 1, 1 - q))), 
         C = as.logical(ifelse(r == 1, rbinom(1, 1, q), 
                               rbinom(1, 1, 1 - q)))) %>% ungroup()
# p: probability that it is raining in Seattle
# r: it is raining in Seattle
# A, B, C: friend A, B, C says it is raining
  # (not to be confused with he or she is telling the truth)
data %>% sample_n(10) %>% print(n = 10) # show 10 of the 21*N rows

Source: local data frame [10 x 5]

       p     r     A     B     C
   <dbl> <lgl> <lgl> <lgl> <lgl>
1   0.70  TRUE  TRUE  TRUE  TRUE
2   0.65  TRUE  TRUE FALSE  TRUE
3   0.85  TRUE  TRUE  TRUE  TRUE
4   0.90  TRUE FALSE  TRUE FALSE
5   0.80  TRUE FALSE  TRUE  TRUE
6   0.75  TRUE  TRUE  TRUE  TRUE
7   0.95  TRUE  TRUE  TRUE FALSE
8   0.70  TRUE FALSE  TRUE  TRUE
9   0.55  TRUE  TRUE FALSE  TRUE
10  0.60 FALSE  TRUE FALSE  TRUE

# Check proportions of r (compared to p)
data %>% group_by(p) %>% summarize(Real_Prob_Rain = mean(r)) %>% 
  rename(Prob_Rain = p) %>% print(n = Inf)

Source: local data frame [21 x 2]

   Prob_Rain Real_Prob_Rain
       <dbl>          <dbl>
1       0.00         0.0000
2       0.05         0.0460
3       0.10         0.0978
4       0.15         0.1494
5       0.20         0.1938
6       0.25         0.2466
7       0.30         0.2956
8       0.35         0.3390
9       0.40         0.3934
10      0.45         0.4524
11      0.50         0.4994
12      0.55         0.5430
13      0.60         0.5900
14      0.65         0.6586
15      0.70         0.6908
16      0.75         0.7406
17      0.80         0.8004
18      0.85         0.8504
19      0.90         0.8942
20      0.95         0.9492
21      1.00         1.0000

# Check proportions of friends telling the truth (compared to q)
data %>% rename(Raining = r) %>% group_by(Raining) %>% 
  summarize("%Cases_All_Friends_Say_Raining" = mean(A + B + C) / 3) %>% 
  print(n = Inf)
data %>% rename(Prob_Rain = p, Raining = r) %>% 
  group_by(Prob_Rain, Raining) %>% 
  summarize("#Cases" = n(), 
            "%Cases_All_Friends_Say_Raining" = mean(A + B + C) / 3) %>% 
  print(n = 11)

Source: local data frame [2 x 2]

  Raining %Cases_All_Friends_Say_Raining
    <lgl>                          <dbl>
1   FALSE                      0.3322547
2    TRUE                      0.6649218

Source: local data frame [40 x 4]
Groups: Prob_Rain [?]

   Prob_Rain Raining #Cases %Cases_All_Friends_Say_Raining
       <dbl>   <lgl>  <int>                          <dbl>
1       0.00   FALSE   5000                      0.3332667
2       0.05   FALSE   4770                      0.3243885
3       0.05    TRUE    230                      0.6826087
4       0.10   FALSE   4511                      0.3302298
5       0.10    TRUE    489                      0.6659850
6       0.15   FALSE   4253                      0.3359981
7       0.15    TRUE    747                      0.6604195
8       0.20   FALSE   4031                      0.3304391
9       0.20    TRUE    969                      0.6584107
10      0.25   FALSE   3767                      0.3342182
11      0.25    TRUE   1233                      0.6580157
..       ...     ...    ...                            ...

# Filter the evidence: all 3 say it rains
# And group by each value of the prior Prob(it rains), p
data_interest <- data %>% filter(A * B * C == TRUE) %>% group_by(p)
# For each value of the prior (p), what is the posterior?
# (i.e., the mean number of cases where it rain, r == 1)
data_interest %>% summarize(Posterior = round(mean(r), 3)) %>% 
  mutate(Posterior_Theoretical = round(8*p / (1 + 7*p), 3)) %>% 
  rename(Prior = p) %>% print(n = Inf)

Source: local data frame [21 x 3]
    
       Prior Posterior Posterior_Theoretical
       <dbl>     <dbl>                 <dbl>
    1   0.00     0.000                 0.000
    2   0.05     0.279                 0.296
    3   0.10     0.498                 0.471
    4   0.15     0.587                 0.585
    5   0.20     0.643                 0.667
    6   0.25     0.705                 0.727
    7   0.30     0.778                 0.774
    8   0.35     0.814                 0.812
    9   0.40     0.845                 0.842
    10  0.45     0.850                 0.867
    11  0.50     0.900                 0.889
    12  0.55     0.903                 0.907
    13  0.60     0.928                 0.923
    14  0.65     0.949                 0.937
    15  0.70     0.933                 0.949
    16  0.75     0.952                 0.960
    17  0.80     0.974                 0.970
    18  0.85     0.977                 0.978
    19  0.90     0.986                 0.986
    20  0.95     0.995                 0.993
    21  1.00     1.000                 1.000

Now let’s consider a more generic case, where not only $p$, but also the number of friends, $N$, and the probability that any of them tells you the truth, $q$, are not fixed. This analysis will give us the morality of this problem, which could be: “Always trust your friends (especially the more you have!), provided that they tell the truth more often than not.”

Without loss of generality, let’s assume that $q$ is a positive rational number, and hence can be written as $a/b$, where $a,b \in \mathbb{N}; b \geq a$. We want to focus on the case where our friends are more likely to tell the truth, so the following condition must hold:

We can write the posterior probability that we want to calculate as:

Since $b/a - 1 \leq 1$, the limit of the posterior probability as $N$ approaches infinity is $1$, regardless of the value of $p$ (for $p > 0$).

I.e., if a sufficiently large number of friends tell you that it is raining in the desert, and the chances that each one of them is messing with you are less than $1/2$, bring an umbrella with you. For $N = 10$ (and $q = 2/3$), the posterior is greater than $0.9$ for a prior as low as $0.009$.

Let’s finish by plotting the posterior against the prior for a few possible values of $q$ and $N$ (our case of interest, $N=3, q=2/3$ corresponds to the light blue line in the upper-right graph). As expected, if $N = 0$ (i.e., we have no evidence), the posterior equals the prior.

posterior_prob <- function(q, N, p) {
  q^N * p / (q^N * p + (1 - q)^N * (1-p))
}
p <- seq(0, 1,  0.01)
library(MASS)
df <- data.frame(Prior = rep(p, each = 24), 
                 N = rep(c(0:3, 5, 10), each = 4), 
                 q = c(0.51, 2/3, 3/4, 4/5)) %>% 
  mutate(Posterior = posterior_prob(q, N, Prior), 
         Friends = as.factor(N), 
         Q = factor(as.character(fractions(q)), 
                    levels = as.character(fractions(unique(q)))))
library(ggplot2)
ggplot(data = df, aes(x = Prior, y = Posterior, colour = Friends)) + 
  geom_line() + 
  scale_color_hue(c = 240) + 
  labs(title = paste('Posterior vs. Prior for 4 possible\nvalues of', 
                     'the (individual) Likelihood')) + 
  facet_wrap( ~ Q, nrow = 2) + coord_fixed()
options(repr.plot.width = 8, repr.plot.height = 8)

Close