Is it good practice to use Linear Least-Squares with SMA?How to correctly apply a linear trendline...
Violin - Can double stops be played when the strings are not next to each other?
Is it true that good novels will automatically sell themselves on Amazon (and so on) and there is no need for one to waste time promoting?
Is it insecure to send a password in a `curl` command?
A single argument pattern definition applies to multiple-argument patterns?
What is a ^ b and (a & b) << 1?
What is the Japanese sound word for the clinking of money?
combinatorics floor summation
Can I use USB data pins as power source
Are Roman Catholic priests ever addressed as pastor
Simplify an interface for flexibly applying rules to periods of time
Why did it take so long to abandon sail after steamships were demonstrated?
Why do passenger jet manufacturers design their planes with stall prevention systems?
Recruiter wants very extensive technical details about all of my previous work
Most cost effective thermostat setting: consistent temperature vs. lowest temperature possible
What is the significance behind "40 days" that often appears in the Bible?
What is "focus distance lower/upper" and how is it different from depth of field?
How to make healing in an exploration game interesting
How do you talk to someone whose loved one is dying?
I am confused as to how the inverse of a certain function is found.
What's the meaning of a knight fighting a snail in medieval book illustrations?
Shortcut for setting origin to vertex
Welcoming 2019 Pi day: How to draw the letter π?
Why no Iridium-level flares from other satellites?
Is it normal that my co-workers at a fitness company criticize my food choices?
Is it good practice to use Linear Least-Squares with SMA?
How to correctly apply a linear trendline equationMeasuring treatment effect on top-ranked subjects selected at point in time from longitudinal dataEnsemble model performs better with worse performing consitutent models?Textbooks on linear regression with least squaresInterpreting regression and $R^2$ with small $n$Solution to force a polynomial curve to end at a specific locationLinear Regression Understanding Least SquaresCan residuals be calculated from N-point moving averages or just the regression line? Also, what is the standard way to determine regression line?Line of best fit does not look like a good fit. Why?Linear least squares algorithms
$begingroup$
I have time-series (daily) data and I want to understand the general trend.
My current approach is:
Calculate the 7-day simple moving average.
Add a line of best fit (linear least squares regression).
Plot, then review metrics such as r, r^2, etc.
Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.
Raw data looks like this:
day + count
2015-01-01 | 123
2015-01-02 | 290
2015-01-03 | 329
2015-01-04 | 276
Let me know if more detail would help- any direction on this is much appreciated.
regression time-series correlation trend moving-average
asked 3 hours ago
Chef36Chef36
61
New contributor
Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
I have time-series (daily) data and I want to understand the general trend.
My current approach is:
Calculate the 7-day simple moving average.
Add a line of best fit (linear least squares regression).
Plot, then review metrics such as r, r^2, etc.
Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.
Raw data looks like this:
day + count
2015-01-01 | 123
2015-01-02 | 290
2015-01-03 | 329
2015-01-04 | 276
Let me know if more detail would help- any direction on this is much appreciated.
regression time-series correlation trend moving-average
asked 3 hours ago
Chef36Chef36
61
New contributor
Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
1
$begingroup$
It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
$endgroup$
– whuber♦
2 hours ago
add a comment |
$begingroup$
I have time-series (daily) data and I want to understand the general trend.
My current approach is:
Calculate the 7-day simple moving average.
Add a line of best fit (linear least squares regression).
Plot, then review metrics such as r, r^2, etc.
Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.
Raw data looks like this:
day + count
2015-01-01 | 123
2015-01-02 | 290
2015-01-03 | 329
2015-01-04 | 276
Let me know if more detail would help- any direction on this is much appreciated.
regression time-series correlation trend moving-average
asked 3 hours ago
Chef36Chef36
61
New contributor
Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
I have time-series (daily) data and I want to understand the general trend.
My current approach is:
Calculate the 7-day simple moving average.
Add a line of best fit (linear least squares regression).
Plot, then review metrics such as r, r^2, etc.
Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.
Raw data looks like this:
day + count
2015-01-01 | 123
2015-01-02 | 290
2015-01-03 | 329
2015-01-04 | 276
Let me know if more detail would help- any direction on this is much appreciated.
regression time-series correlation trend moving-average
regression time-series correlation trend moving-average
asked 3 hours ago
Chef36Chef36
61
New contributor
Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 3 hours ago
Chef36Chef36
61
New contributor
Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 3 hours ago
Chef36Chef36
61
New contributor
Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 3 hours ago
Chef36Chef36
61
asked 3 hours ago
Chef36Chef36
61
61
New contributor
Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
1
$begingroup$
It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
$endgroup$
– whuber♦
2 hours ago
add a comment |
1
$begingroup$
It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
$endgroup$
– whuber♦
2 hours ago
1
1
$begingroup$
It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
$endgroup$
– whuber♦
2 hours ago
$begingroup$
It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
$endgroup$
– whuber♦
2 hours ago
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.
Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.
These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.
answered 2 hours ago
Gkhan CebsGkhan Cebs
1443
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "65"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Chef36 is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f397917%2fis-it-good-practice-to-use-linear-least-squares-with-sma%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.
Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.
These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.
answered 2 hours ago
Gkhan CebsGkhan Cebs
1443
$endgroup$
add a comment |
$begingroup$
Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.
Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.
These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.
answered 2 hours ago
Gkhan CebsGkhan Cebs
1443
$endgroup$
add a comment |
$begingroup$
Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.
Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.
These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.
answered 2 hours ago
Gkhan CebsGkhan Cebs
1443
$endgroup$
Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.
Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.
These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.
answered 2 hours ago
Gkhan CebsGkhan Cebs
1443
answered 2 hours ago
Gkhan CebsGkhan Cebs
1443
answered 2 hours ago
Gkhan CebsGkhan Cebs
1443
answered 2 hours ago
Gkhan CebsGkhan Cebs
1443
1443
add a comment |
add a comment |
Chef36 is a new contributor. Be nice, and check out our Code of Conduct.
Chef36 is a new contributor. Be nice, and check out our Code of Conduct.
Chef36 is a new contributor. Be nice, and check out our Code of Conduct.
Chef36 is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Cross Validated!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f397917%2fis-it-good-practice-to-use-linear-least-squares-with-sma%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
$begingroup$
It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
$endgroup$
– whuber♦
2 hours ago