The console should have the elevation set to the true elevation, and the elevation provided to CWOP at signup should be true as well. Altimeter should be sent for pressure to CWOP, using the feature for doing that in WeatherDisplay. If this is not done, and you are not near sea level, then you can expect to see your barometer drift when compared to reliable stations (i.e. those operated by the NWS & FAA).
The quality checks are useful for alerting people to possible problems, but the expected values take into consideration all the nearby stations, including other PWS that may be providing bad data themselves. So it should not be relied upon as something to which you calibrate your instruments. It is a useful in that it indicates you might look into the issue to see if it is in fact your instrument that is off. For barometer, where microclimate is not much of an issue, I like to use the check boxes at the bottom of the graph to select the nearest two NWS stations to include in the graph. Those are what you should be comparing to to see if you have actual drift. If you have set your elevations properly, and are submitting altimeter as you should, then what you may find is that your pressure tracks well with the NWS stations, but it is the quality check expected value is drifting because of other nearby PWS that are NOT sending proper values for pressure.
For other sensors like temperature/humidity/dewpoint you really have to take the quality check numbers with a grain of salt. Being out of sorts according to the quality check might indicate a problem, but it is quite a bit more likely that it's caused by microclimate variance. For example, during the spring, fall and winter, my dew point quality check is excellent, and I track well to the nearest airport ASOS station (about 8-10 miles away) as well. But in the summer I get a big thumbs down because my humidity reads lower than expected. But I don't worry about it too much because all the nearby PWSs and the ASOS station are in the middle of the big, wide, heavily irrigated Boise river valley. I am on the edge of the valley on dry land, with miles of unirrigated BLM land to the north. Interestingly, my humidity does track much more closely to the Boise airport 20 miles away, and it, like me, is on dry land at the edge of dry BLM land. So I just let it give me a thumbs down. I looked into the issue, and decided it was microclimate, and I never calibrate my temperature or humidity because of it.
Steve