Gradient Descent is an algorithm for finding the minimum of a function. It iteratively calculates partial derivatives (gradients) of the function and descends in steps proportional to those partial derivatives. One major application of Gradient Descent is fitting a parameterized model to a set of ...

**-2**

votes

**2**answers

37 views

### Why that I have written for Andrew Ng's course not accepted?

Andrew Ng's course in Coursera, which Stanford's Machine Learning course, features programming assignments that deal with implementing the algorithms taught in class. The goal of this assignment is to ...

**0**

votes

**1**answer

31 views

### Neural Network converges on answer, then oscillates wildly

In trying to understand fully connected ANN's, I'm starting with a simple 2-D linear regression example. My network is trivial - one input layer and an output layer with a set of weights between ...

**0**

votes

**1**answer

18 views

### tensorflow GradientDescentOptimizer: Incompatible shapes between op input and calculated input gradient

The model worked well before optimization step. However, when I want to optimize my model, the error message showed up: Incompatible shapes between op input and calculated input gradient. ...

**1**

vote

**1**answer

29 views

### python **kwags Error: function takes 6 positional arguments but 8 were given

When I was trying to develop a gradient descent, I discovered an interesting problem that I cannot use **kwargs effectively. My function looks like def gradient_descent(g,x,y,alpha,max_its,w,**...

**0**

votes

**0**answers

12 views

### Is Gradient Boosted Tree boosting on the residuals or on the complete training set?

Please note, this question is a duplicate on stats.stackexchange.com.I'm just not getting an answer there. So I'll delete the unanswered and keep the answered as soon as I get the first answer.It ...

**-1**

votes

**1**answer

26 views

### Relationship logistic regression and Stochastic gradient descent In Formula [closed]

Considering in Formula term, from my opinion that SGD apply to final result of Logistic regression. Not sure whether correct or notjust wondering the relationship between stochastic gradient descent ...

**0**

votes

**0**answers

25 views

### tensorflow logistic regression train by GradientDescentOptimizer return nan

trying to implement logistic regression with tensorflow in python. using "maximum likelihood method" cost (maximum * -1 to fit gradient_descent)as the loss function;GradientDescentOptimizer as ...

**0**

votes

**1**answer

15 views

### Stochastic Gradient Descent increases Cost Function

In Neural Networks, Gradient Descent looks over the entire training set in order to calculate gradient. The cost function decreases over iterations. If cost function increases, it is usually because ...

**1**

vote

**1**answer

30 views

### Backprop and forward pass in neural networks

I am trying to design a simple a neural network but I am facing some problems.My output keeps converging to 0.5 when I use ReLU and to 0.7 when I use sigmoid function.Please suggest me :Is there ...

**-1**

votes

**1**answer

95 views

### Why do we need to change bias in neural network gradient descent?

I'm implementing gradient descent in a neural network, and I'm having trouble understanding why we need to find the derivative with respect to all thetas and biases, since if we perform gradient ...

**0**

votes

**1**answer

36 views

### Stochastic Gradient Descent for Linear Regression on partial derivatives

I am implementing stochastic gradient descent for linear regression manually by considering the partial derivatives (df/dm) and (df/db)The objective is we have to randomly select the w0(weights) and ...

**1**

vote

**1**answer

18 views

### What is the error term in backpropagation through time if I only have one output?

In this question, RNN: Back-propagation through time when output is taken only at final timestep I've seen that if I only have one output at final time step T, which is y(T), then the error at earlier ...

**0**

votes

**1**answer

23 views

### Neural network in Theano with sparse weight matrix diverging

I am working on a neural network with very large sparse weight matrices. The zero values in the weights are to remain zero and not be changed. Gradients should only be calculated and propagated for ...

**-1**

votes

**0**answers

19 views

### Gradient descent with multiple variables

I am taking Andrew Ng's course, and I'm trying to create a gradient descent algorithm to find an optimal theta. I am given these variables:alpha (learning rate)=0.01X (examples x features matrix)...

**0**

votes

**1**answer

28 views

### Is there a Python library where I can import a gradient descent function/method?

One way to do gradient descent in Python is to code it myself. However, given how popular a concept it is in machine learning, I was wondering if there is a Python library that I can import that gives ...