Recently encountered an error. going to Home >> Security Center >> ModSecurity >> Vendors and clicking Install. However I’m getting the following error messages:

Error:The system experienced the following error when it attempted to install the ?OWASP ModSecurity Core Rule Set V3.0? vendor: API failure: The system could not validate the new Apache configuration because httpd exited with a nonzero value. 

Fix:

Backup the /var/cpanel/modsec_cpanel_conf_datastore file.

Manually edit /var/cpanel/modsec_cpanel_conf_datastore file with your favorite editor. Removed all active configs, active vendors, disabled rules, and updates. Just keep disabled rule. Save the file.

Login to WHM >> Home >> Security Center >> Modsecurity Vendors >> Manage Vendors >> OWASP ModSecurity Core Rule Set >> click +install

Modsecurity rules will should be installed without any issue.

Other links:
https://forums.cpanel.net/threads/mod_security-and-easyapache-4.501871/