The point of the story is that, despite Aylmer’s obsession, Georgiana was already perfect as a partner, at least in Hawthorne’s view. She was loyal and supportive, willing to do anything for his sake, even drink poison, as she explained earlier in the story. However, instead of living happily with her, he became obsessed with the earthly phenomenon of the birthmark instead of recognizing Georgiana’s spiritual nobility. With her beauty, she was already as close to perfection as possible, but Aylmer failed to see that and focused on the unachievable. Overwhelmed by his obsession, he sacrificed his chance at lifelong happiness for a moment of triumph followed by regret, hence the concluding phrase.