It was probably foolhardy for you to give him de facto custody, as he can now move to have the court order modified to reflect that reality & then sue you for child support. In any event, if continued informal methods do not work, then yes, you’d need to file an enforcement petition in court. Schedule a consult with a Bronx Child Custody attorney for a full assessment.