Matplotlib Test Causing Failures

[WillAyd]

ref #22039 where it started there appear to be a lot of failures for Travis / Azure with the below test:

    def test_matplotlib_scatter_datetime64(self):
        # https://github.com/matplotlib/matplotlib/issues/11391
        df = DataFrame(np.random.RandomState(0).rand(10, 2),
                       columns=["x", "y"])
        df["time"] = date_range("2018-01-01", periods=10, freq="D")
        fig, ax = self.plt.subplots()
        ax.scatter(x="time", y="y", data=df)
        fig.canvas.draw()
        label = ax.get_xticklabels()[0]
>       assert label.get_text() == '2017-12-12'
E       AssertionError: assert '2017-12-08' == '2017-12-12'
E         - 2017-12-08
E         ?         ^^
E         + 2017-12-12
E         ?         ^^

I wasn't able to reproduce locally so may be an issue with matplotlib versioning, though it's questionable if the intention of the test above really is to ensure the first date on the X axis or if that was just an arbitrary check

[TomAugspurger]

#22790 I think.

I'll get to this in a few hours unless someone beats me to it.