In this tidbit I want to share some tips on how to troubleshoot FMC policy deployment errors. It is important to understand that the policy deployment is broken down into the following phases:
Phase 0--Deployment Initialization
Phase 1--Database Object Collection
Phase 2--Policy and Object Collection
Phase 3--NGFW Command Line Configuration Generation
Phase 4--Device Deployment Package Generation
Phase 5--Sending and Receiving the Deployment Package
Phase 6--Pending Deployment, Deployment Actions, and Deployment Success Messages
The two most commonly referenced logs are policy_deployment.log and usmsharedsvcs.log. The /var/opt/CSCOpx/MDC/log/operation/usmsharedsvcs.log shows the following:
Marks the beginning of policy deployment task
The completion of each phase, which aides in determining what phase failed
The /var/log/sf/policy_deployment.log shows the following:
Describes detailed steps taken to build the deployment packages
Best used to troubleshoot Phase 1-4 issues
Common policy deployment errors seen in FMC:
device_currently_under_deployment -- Deployment failed due to another deployment in progress for this device. Try again later. FIX: Wait for existing deployment to finish.
failed_to_retrieve_running_configuration -- Deployment failed due to failure retrieving running configuration information from device. Retry deployment. FIX: This message can occur when connectivity between an end sensor and an FMC is not working as expected. Verify the tunnel health between the units and monitor the connectivity between the two devices.
device_is_busy -- Deployment failed as device may be running a previous deployment or restarting. If problem persists after retrying later, contact Cisco TAC. FIX: This message is shown, when FMC attempts a deploy, while a previous deployment is in progress on FTD. Typically happens when a previous deployment is unfinished on FTD and the FTD rebooted or the ngfwManager process on FTD restarted. A retry after 20 minutes to allow processes to formally timeout should resolve this issue.
device_failure_timeout -- Deployment to device failed due to timeout. Retry deployment. FIX: This is related to FTD deployment. Processes on FTD wait 30 minutes for the dispatch to complete deployment. If not, it times out. If this occurs, verify inter-device connectivity and if the connectivity is as expected,.
That wraps up this tidbit on troubleshooting FMC policy deployment errors. Cheers!