Subscribe to DSC Newsletter

Multi-variable Linear Regression: optimize theta by using BFGS algorithm.

Hello Everyone
I want to switch my career in Data Science and have been learning Machine Learning since last two weeks. I want to learn optimization technique and applying the BFGS algorithm on some data to get optimize value of theta in linear regression.
When I'm running my code in python, it gives the following error:

> derphi0 = np.dot(gfk, pk)
> ValueError: shapes (3,47) and (3,47) not aligned: 47 (dim 1) != 3 (dim0)

Any help will be appreciated.

Following is my python code for the same:

import numpy as np
from scipy.optimize import minimize
import matplotlib.pyplot as plt


filename='ex1data2.txt'
data = np.loadtxt(filename, delimiter=',')

X=data[:,0:2]; y=data[:,-1]
m=len(y)
X1=X.reshape((m, 2))
y1=y.reshape((m, 1))

X_norm = np.copy(X1)
mu = np.zeros((1, len(X1[1,:])))
sigma = np.zeros((1, len(X1[1,:])))
mu = np.mean(X_norm,axis=0)
mu=np.array(mu)
sigma = np.std(X_norm,axis=0)
sigma=np.array(sigma)

for iter in range(0,(len(X1[1,:]))):
X_norm[:,iter] = (X1[:,iter] - mu[iter])/sigma[iter]

X=np.concatenate([np.ones((m,1)), X_norm], axis=1)
theta0=np.zeros((3,1))

def cost(theta):
    h=(np.dot(X,theta)-y1)**2
    return (h.sum(axis=0))/(2*m);

def gradient(theta):
    h1=np.dot(X,theta)-y1
    grad=(np.dot(X.T,h1))/m
    return grad;

thetaopt = minimize(cost, theta0, method='BFGS', jac=gradient,options={'disp': True,'maxiter':400})
print(thetaopt)

I also, have attached the data set that I'm using in my code.

Any kind of help will be appreciated.
Thanks

Views: 586

Attachments:

Reply to This

Replies to This Discussion

I don't understand what you are doing because I am new to these techniques, but I am a python programmer.

lets us solve line by line- Below is some part of your code(I did some changes to your script I believe some of your lines are redundant). your "for" loop wont work because "mu" and "sigma" are of size 2 where as "iter" will go up to 46

import numpy as np
from scipy.optimize import minimize
import matplotlib.pyplot as plt

filename='ex1data2.txt'
data = np.loadtxt(filename, delimiter=',')

X=data[:,0:2]; Y=data[:,-1]
X_norm = np.copy(X)
mu = np.mean(X_norm,axis=0) #column wise
sigma = np.std(X_norm,axis=0)#column wise

for iter in range(0,(len(X))):
    X_norm[:,iter] = (X[:,iter] - mu[iter])/sigma[iter]

Reply to Discussion

RSS

Videos

  • Add Videos
  • View All

Follow Us

© 2018   Data Science Central ®   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service