After enabling renovatebot on an org for the first time, it doesn't seem like there is a way to prevent opening a PR in every repo.
Using the gh cli, we can retrieve every repo, collect the PR number associated with the new PR, and close each one until we're ready to configure each.
ORG=myorg
gh repo list $ORG -L 400 --json name --jq '.[].name' > repo-list.text
cat repo-list.txt | while read repo; do \
echo gh pr close -R $ORG/$repo $(gh pr list -R $ORG/$repo --json title,number --jq '.[] | select(.title == "Configure Renovate") | .number')
done