The rocker introduced the on-again, off-again couple’s “little celestial seed” via Instagram on Thursday with a sweet video holding her hand.