PerfBeacon was initially built to monitor webpages on a schedule.
However if you only want to test your site when it gets deployed, or whenever a change is made, that's possible too.
Settings
tab. Test Frequency
dropdown, and select "Never", as below: