Vjylku

Gaining more land

How to resolve: Reviewer #1 says remove section X vs. Reviewer #2 says expand section X

Professor forcing me to attend a conference, I can't afford even with 50% funding

Is a piano played in the same way as a harmonium?

What is better: yes / no radio, or simple checkbox?

In the late 1940’s to early 1950’s what technology was available that could melt ice?

How do we create new idioms and use them in a novel?

When Schnorr signatures are part of Bitcoin will it be possible validate each block with only one signature validation?

Virginia employer terminated employee and wants signing bonus returned

Is this Paypal Github SDK reference really a dangerous site?

Vocabulary for giving just numbers, not a full answer

Why do we say ‘pairwise disjoint’, rather than ‘disjoint’?

ER diagram relationship node size adjustment

What will happen if my luggage gets delayed?

Signed and unsigned numbers

Getting the || sign while using Kurier

Can one live in the U.S. and not use a credit card?

How to design an organic heat-shield?

What is Tony Stark injecting into himself in Iron Man 3?

PTIJ: Why does only a Shor Tam ask at the Seder, and not a Shor Mu'ad?

QQ Plot and Shapiro Wilk Test Disagree

Should I take out a loan for a friend to invest on my behalf?

Power Strip for Europe

Does the phrase "more obvious" mean "just barely noticeable among many" in certain contexts?

QQ Plot and Shapiro Wilk Test Disagree

What is the meaning of “All models are wrong, but some are useful”Residual plot with a slope of 1What to do when Kolmogorov-Smirnov test is significant for residuals of parametric test but skewness and kurtosis look normal?Heteroskedasticity and residuals normalityLinear regresson lm or stepwise regression here using R?How incorrect is a regression model when assumptions are not met?Residuals analysis: interpretation of a scatter plotNormal Q-Q plot - to check AssumptionHow should I transform the qqplots such that the distribution will be normal?residual plot for positive variableLinear Regression on Boston Housing Price?

1

$begingroup$

My QQ Plot shows that the data is not normally distributed

qqplot(residual_values, fit = True, line = '45')

pylab.show()

It has a skewness of 0.54

residual_values.skew()  # 0.5469389365591185

But the p_value of Shapiro test is greater than 0.05, telling me that it is normally distributed

shapiro(residual_values) # (0.9569438099861145, 0.2261517345905304)

What is the correct inference from this, Is it Normally Distributed or not?

regression machine-learning

share|cite|improve this question

asked 5 hours ago

ShinigamiShinigami

164

New contributor

Shinigami is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

• 
  
  
  

  
  3
  

  
  
  
  
  
  

  
  $begingroup$
  The QQ plot looks consistent with being normally distributed. Did you expect every point to fall exactly on the line?
  $endgroup$
  – The Laconic
  3 hours ago
  

  
  
  
  


• 
  
  
  

  
  3
  

  
  
  
  
  
  

  
  $begingroup$
  It is approximately normally distributed if you are prepared to discount slight skewness. No procedure ever indicates more.
  $endgroup$
  – Nick Cox
  2 hours ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @TheLaconic Sorry I am new to ML.
  $endgroup$
  – Shinigami
  2 hours ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @NickCox Thank you Sorry reputation is low to upvote you guys
  $endgroup$
  – Shinigami
  2 hours ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  It's approximately normal, the skewness in the sample is quite mild; this doesn't automatically mean the population is also skewed (though I expect it is). A high p-value on a test of normality doesn't mean that it is normal, only that you couldn't detect whatever population non-normality there was. (The answer to "is it normally distributed" is "no" - unless you generated it to be normal it won't actually be normal -- but why would it have to be?)
  $endgroup$
  – Glen_b♦
  17 mins ago
  
  
  

  
  
  
  

add a comment | 

1

$begingroup$

My QQ Plot shows that the data is not normally distributed

qqplot(residual_values, fit = True, line = '45')

pylab.show()

It has a skewness of 0.54

residual_values.skew()  # 0.5469389365591185

But the p_value of Shapiro test is greater than 0.05, telling me that it is normally distributed

shapiro(residual_values) # (0.9569438099861145, 0.2261517345905304)

What is the correct inference from this, Is it Normally Distributed or not?

regression machine-learning

share|cite|improve this question

asked 5 hours ago

ShinigamiShinigami

164

New contributor

Shinigami is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

• 
  
  
  

  
  3
  

  
  
  
  
  
  

  
  $begingroup$
  The QQ plot looks consistent with being normally distributed. Did you expect every point to fall exactly on the line?
  $endgroup$
  – The Laconic
  3 hours ago
  

  
  
  
  


• 
  
  
  

  
  3
  

  
  
  
  
  
  

  
  $begingroup$
  It is approximately normally distributed if you are prepared to discount slight skewness. No procedure ever indicates more.
  $endgroup$
  – Nick Cox
  2 hours ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @TheLaconic Sorry I am new to ML.
  $endgroup$
  – Shinigami
  2 hours ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @NickCox Thank you Sorry reputation is low to upvote you guys
  $endgroup$
  – Shinigami
  2 hours ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  It's approximately normal, the skewness in the sample is quite mild; this doesn't automatically mean the population is also skewed (though I expect it is). A high p-value on a test of normality doesn't mean that it is normal, only that you couldn't detect whatever population non-normality there was. (The answer to "is it normally distributed" is "no" - unless you generated it to be normal it won't actually be normal -- but why would it have to be?)
  $endgroup$
  – Glen_b♦
  17 mins ago
  
  
  

  
  
  
  

add a comment | 

$begingroup$

My QQ Plot shows that the data is not normally distributed

qqplot(residual_values, fit = True, line = '45')

pylab.show()

It has a skewness of 0.54

residual_values.skew()  # 0.5469389365591185

But the p_value of Shapiro test is greater than 0.05, telling me that it is normally distributed

shapiro(residual_values) # (0.9569438099861145, 0.2261517345905304)

What is the correct inference from this, Is it Normally Distributed or not?

regression machine-learning

share|cite|improve this question

asked 5 hours ago

ShinigamiShinigami

164

New contributor

Shinigami is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

qqplot(residual_values, fit = True, line = '45')

pylab.show()

residual_values.skew()  # 0.5469389365591185

shapiro(residual_values) # (0.9569438099861145, 0.2261517345905304)

share|cite|improve this question

asked 5 hours ago

ShinigamiShinigami

164

New contributor

Shinigami is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

share|cite|improve this question

asked 5 hours ago

ShinigamiShinigami

164

New contributor

Shinigami is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 5 hours ago

ShinigamiShinigami

164

New contributor

Shinigami is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 5 hours ago

ShinigamiShinigami

164

4 Answers
4

active

oldest

votes

1

$begingroup$

The QQ plot is an informal test of normality that can give you some insight into the nature of deviations from normality; for example, whether the distribution has some skew, or fat tails, or whether there are specific observations that deviate from what you would expect from a normal distribution (outliers). The QQ plot can often convince you that the distribution is definitely not normal, but this isn't such a case. Here, the points fall more or less along the line, which is broadly consistent with normality--intuitively, the sort of variation you would expect to see in a small sample.

The Shapiro test is a formal test of normality. I'm not familiar with the shapiro function's output, so I'm not sure which number, if either, is supposed to be the p-value, but if you say it's largish, then we are led to accept the null hypothesis of normality. And this is consistent with what we see qualitatively in the QQ plot.

share|cite|improve this answer

answered 1 hour ago

The LaconicThe Laconic

1,0741615

$endgroup$

add a comment | 

1

$begingroup$

The q-q is consistent with (not "proving") approximate normality, more or less.

The Shapiro-Wilk is a formal test of normality and as such, it cannot confirm the null hypothesis of normality. The data may be reasonably consistent with normality yet still be from a different nonnormal underlying distribution. Frequentist hypothesis tests, as a general rule, cannot prove a hypothesis, and failure to reject (p>alpha) does not support the null hypothesis.

@The Laconic gave some decent advice to interpret the q-q plot. However, large p-values do not lead you to accept the null hypothesis (therefore, you don't conclude normality based on this test; the best you can do is say insufficient evidence of nonnormality at the a priori chosen alpha level).

share|cite|improve this answer

answered 1 hour ago

LSCLSC

1707

$endgroup$

add a comment | 

1

$begingroup$

The shapiro-wilk p-value being >0.05 indicates lack of evidence to against normality. That is consistent with the QQ plot you showed, which is not too far off the line. I don't see what the inconsistency is here. Also, you should give a CI for the skewness coefficient.

share|cite|improve this answer

answered 1 hour ago

beta1_equals_beta2beta1_equals_beta2

512

$endgroup$

add a comment | 

0

$begingroup$

My understanding is that, given power issues with normality tests, they are not highly recommended. As a result I don't use them any more, preferring QQ plots (which are recommended in the literature I have seen).

share|cite|improve this answer

answered 31 mins ago

user54285user54285

413

$endgroup$

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "65"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});

Shinigami is a new contributor. Be nice, and check out our Code of Conduct.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f396717%2fqq-plot-and-shapiro-wilk-test-disagree%23new-answer', 'question_page');
}
);

Post as a guest

Name

Email

Required, but never shown

1

$begingroup$

The QQ plot is an informal test of normality that can give you some insight into the nature of deviations from normality; for example, whether the distribution has some skew, or fat tails, or whether there are specific observations that deviate from what you would expect from a normal distribution (outliers). The QQ plot can often convince you that the distribution is definitely not normal, but this isn't such a case. Here, the points fall more or less along the line, which is broadly consistent with normality--intuitively, the sort of variation you would expect to see in a small sample.

The Shapiro test is a formal test of normality. I'm not familiar with the shapiro function's output, so I'm not sure which number, if either, is supposed to be the p-value, but if you say it's largish, then we are led to accept the null hypothesis of normality. And this is consistent with what we see qualitatively in the QQ plot.

share|cite|improve this answer

answered 1 hour ago

The LaconicThe Laconic

1,0741615

$endgroup$

add a comment | 

1

$begingroup$

The QQ plot is an informal test of normality that can give you some insight into the nature of deviations from normality; for example, whether the distribution has some skew, or fat tails, or whether there are specific observations that deviate from what you would expect from a normal distribution (outliers). The QQ plot can often convince you that the distribution is definitely not normal, but this isn't such a case. Here, the points fall more or less along the line, which is broadly consistent with normality--intuitively, the sort of variation you would expect to see in a small sample.

The Shapiro test is a formal test of normality. I'm not familiar with the shapiro function's output, so I'm not sure which number, if either, is supposed to be the p-value, but if you say it's largish, then we are led to accept the null hypothesis of normality. And this is consistent with what we see qualitatively in the QQ plot.

share|cite|improve this answer

answered 1 hour ago

The LaconicThe Laconic

1,0741615

$endgroup$

add a comment | 

$begingroup$

The QQ plot is an informal test of normality that can give you some insight into the nature of deviations from normality; for example, whether the distribution has some skew, or fat tails, or whether there are specific observations that deviate from what you would expect from a normal distribution (outliers). The QQ plot can often convince you that the distribution is definitely not normal, but this isn't such a case. Here, the points fall more or less along the line, which is broadly consistent with normality--intuitively, the sort of variation you would expect to see in a small sample.

The Shapiro test is a formal test of normality. I'm not familiar with the shapiro function's output, so I'm not sure which number, if either, is supposed to be the p-value, but if you say it's largish, then we are led to accept the null hypothesis of normality. And this is consistent with what we see qualitatively in the QQ plot.

share|cite|improve this answer

answered 1 hour ago

The LaconicThe Laconic

1,0741615

$endgroup$

share|cite|improve this answer

answered 1 hour ago

The LaconicThe Laconic

1,0741615

answered 1 hour ago

The LaconicThe Laconic

1,0741615

answered 1 hour ago

The LaconicThe Laconic

1,0741615

1

$begingroup$

The q-q is consistent with (not "proving") approximate normality, more or less.

The Shapiro-Wilk is a formal test of normality and as such, it cannot confirm the null hypothesis of normality. The data may be reasonably consistent with normality yet still be from a different nonnormal underlying distribution. Frequentist hypothesis tests, as a general rule, cannot prove a hypothesis, and failure to reject (p>alpha) does not support the null hypothesis.

@The Laconic gave some decent advice to interpret the q-q plot. However, large p-values do not lead you to accept the null hypothesis (therefore, you don't conclude normality based on this test; the best you can do is say insufficient evidence of nonnormality at the a priori chosen alpha level).

share|cite|improve this answer

answered 1 hour ago

LSCLSC

1707

$endgroup$

add a comment | 

1

$begingroup$

The q-q is consistent with (not "proving") approximate normality, more or less.

The Shapiro-Wilk is a formal test of normality and as such, it cannot confirm the null hypothesis of normality. The data may be reasonably consistent with normality yet still be from a different nonnormal underlying distribution. Frequentist hypothesis tests, as a general rule, cannot prove a hypothesis, and failure to reject (p>alpha) does not support the null hypothesis.

@The Laconic gave some decent advice to interpret the q-q plot. However, large p-values do not lead you to accept the null hypothesis (therefore, you don't conclude normality based on this test; the best you can do is say insufficient evidence of nonnormality at the a priori chosen alpha level).

share|cite|improve this answer

answered 1 hour ago

LSCLSC

1707

$endgroup$

add a comment | 

$begingroup$

The q-q is consistent with (not "proving") approximate normality, more or less.

The Shapiro-Wilk is a formal test of normality and as such, it cannot confirm the null hypothesis of normality. The data may be reasonably consistent with normality yet still be from a different nonnormal underlying distribution. Frequentist hypothesis tests, as a general rule, cannot prove a hypothesis, and failure to reject (p>alpha) does not support the null hypothesis.

@The Laconic gave some decent advice to interpret the q-q plot. However, large p-values do not lead you to accept the null hypothesis (therefore, you don't conclude normality based on this test; the best you can do is say insufficient evidence of nonnormality at the a priori chosen alpha level).

share|cite|improve this answer

answered 1 hour ago

LSCLSC

1707

$endgroup$

share|cite|improve this answer

answered 1 hour ago

LSCLSC

1707

answered 1 hour ago

LSCLSC

1707

answered 1 hour ago

LSCLSC

1707

1

$begingroup$

The shapiro-wilk p-value being >0.05 indicates lack of evidence to against normality. That is consistent with the QQ plot you showed, which is not too far off the line. I don't see what the inconsistency is here. Also, you should give a CI for the skewness coefficient.

share|cite|improve this answer

answered 1 hour ago

beta1_equals_beta2beta1_equals_beta2

512

$endgroup$

add a comment | 

1

$begingroup$

The shapiro-wilk p-value being >0.05 indicates lack of evidence to against normality. That is consistent with the QQ plot you showed, which is not too far off the line. I don't see what the inconsistency is here. Also, you should give a CI for the skewness coefficient.

share|cite|improve this answer

answered 1 hour ago

beta1_equals_beta2beta1_equals_beta2

512

$endgroup$

add a comment | 

$begingroup$

The shapiro-wilk p-value being >0.05 indicates lack of evidence to against normality. That is consistent with the QQ plot you showed, which is not too far off the line. I don't see what the inconsistency is here. Also, you should give a CI for the skewness coefficient.

share|cite|improve this answer

answered 1 hour ago

beta1_equals_beta2beta1_equals_beta2

512

$endgroup$

share|cite|improve this answer

answered 1 hour ago

beta1_equals_beta2beta1_equals_beta2

512

answered 1 hour ago

beta1_equals_beta2beta1_equals_beta2

512

answered 1 hour ago

beta1_equals_beta2beta1_equals_beta2

512

0

$begingroup$

My understanding is that, given power issues with normality tests, they are not highly recommended. As a result I don't use them any more, preferring QQ plots (which are recommended in the literature I have seen).

share|cite|improve this answer

answered 31 mins ago

user54285user54285

413

$endgroup$

add a comment | 

0

$begingroup$

My understanding is that, given power issues with normality tests, they are not highly recommended. As a result I don't use them any more, preferring QQ plots (which are recommended in the literature I have seen).

share|cite|improve this answer

answered 31 mins ago

user54285user54285

413

$endgroup$

add a comment | 

$begingroup$

My understanding is that, given power issues with normality tests, they are not highly recommended. As a result I don't use them any more, preferring QQ plots (which are recommended in the literature I have seen).

share|cite|improve this answer

answered 31 mins ago

user54285user54285

413

$endgroup$

share|cite|improve this answer

answered 31 mins ago

user54285user54285

413

answered 31 mins ago

user54285user54285

413

answered 31 mins ago

user54285user54285

413