Do I need to leave Design Palette Pro installed after I make my changes?

Yes.  Design Palette Pro generates a custom CSS file that is independent of the child theme, and will only load that file when the plugin is activated.  For the Design Palette customizations to display on the site, the plugin needs to remain activated.

If want to deactivate the plugin but keep the customizations, you might want to try our free CSS Export add-on.