I needed to add the failover key <KEY> command in a Cisco ASA firewall pair. The failover pair is working but the previous admin forgot to add this command. I used two Cisco ASAv firewalls in my GNS3 lab to simulate if it's relatively safe to add the said command and wouldn't break the failover pair. It's still advisable to do this in an approved maintenance window.
ASAv-1# show version
Cisco Adaptive
Security Appliance Software Version 9.8(1)
Firepower Extensible
Operating System Version 2.2(1.47)
Device Manager
Version 7.8(1)
Compiled on Wed
10-May-17 15:38 PDT by builders
System image file is
"boot:/asa981-smp-k8.bin"
Config file at boot
was "startup-config"
ASAv-1 up 16 mins 35
secs
Hardware: ASAv, 2048 MB RAM, CPU Pentium II 1900 MHz,
Model Id: ASAv10
Internal ATA Compact
Flash, 8192MB
Slot 1: ATA Compact
Flash, 8192MB
BIOS Flash Firmware
Hub @ 0x0, 0KB
0: Ext: Management0/0 : address is 0ce5.a655.f500, irq 11
1: Ext: GigabitEthernet0/0 : address is 0ce5.a655.f501, irq 11
2: Ext: GigabitEthernet0/1 : address is 0ce5.a655.f502, irq 10
3: Ext: GigabitEthernet0/2 : address is 0ce5.a655.f503, irq 10
4: Ext: GigabitEthernet0/3 : address is 0ce5.a655.f504, irq 11
5: Ext: GigabitEthernet0/4 : address is 0ce5.a655.f505, irq 11
6: Ext: GigabitEthernet0/5 : address is 0ce5.a655.f506, irq 10
7: Ext: GigabitEthernet0/6 : address is 0ce5.a655.f507, irq 10
License mode: Smart
Licensing
ASAv Platform
License State: Unlicensed
No active
entitlement: no feature tier and no throughput level configured
*Memory resource
allocation is more than the permitted limit.
Licensed features
for this platform:
Maximum VLANs : 50
Inside Hosts : Unlimited
Failover : Active/Standby // ASAv SUPPORTS ACTIVE/STANDBY BY DEFAULT
Encryption-DES : Enabled
Encryption-3DES-AES : Enabled
Security
Contexts : 0
Carrier : Disabled
AnyConnect Premium
Peers : 2
AnyConnect
Essentials : Disabled
Other VPN Peers : 250
Total VPN Peers : 250
AnyConnect for
Mobile : Disabled
AnyConnect for Cisco
VPN Phone : Disabled
Advanced Endpoint
Assessment : Disabled
Shared License : Disabled
Total TLS Proxy
Sessions : 2
Botnet Traffic
Filter : Enabled
Cluster : Disabled
Serial Number:
9A81V5LKN5F
Image type : Release
Key version : A
Configuration last
modified by enable_15 at 03:03:53.969 UTC Tue Jan 25 2022
-----
ASAv-2# show version
Cisco Adaptive
Security Appliance Software Version 9.8(1)
Firepower Extensible
Operating System Version 2.2(1.47)
Device Manager
Version 7.8(1)
Compiled on Wed
10-May-17 15:38 PDT by builders
System image file is
"boot:/asa981-smp-k8.bin"
Config file at boot
was "startup-config"
ASAv-2 up 1 min 53
secs
Hardware: ASAv, 2048 MB RAM, CPU Pentium II 1900 MHz,
Model Id: ASAv10
Internal ATA Compact
Flash, 8192MB
Slot 1: ATA Compact
Flash, 8192MB
BIOS Flash Firmware
Hub @ 0x0, 0KB
0: Ext: Management0/0 : address is 0ce5.a6d4.2800, irq 11
1: Ext: GigabitEthernet0/0 : address is 0ce5.a6d4.2801, irq 11
2: Ext: GigabitEthernet0/1 : address is 0ce5.a6d4.2802, irq 10
3: Ext: GigabitEthernet0/2 : address is 0ce5.a6d4.2803, irq 10
4: Ext: GigabitEthernet0/3 : address is 0ce5.a6d4.2804, irq 11
5: Ext: GigabitEthernet0/4 : address is 0ce5.a6d4.2805, irq 11
6: Ext: GigabitEthernet0/5 : address is 0ce5.a6d4.2806, irq 10
7: Ext: GigabitEthernet0/6 : address is 0ce5.a6d4.2807, irq 10
License mode: Smart
Licensing
ASAv Platform
License State: Unlicensed
No active
entitlement: no feature tier and no throughput level configured
*Memory resource
allocation is more than the permitted limit.
Licensed features
for this platform:
Maximum VLANs : 50
Inside Hosts : Unlimited
Failover : Active/Standby
Encryption-DES : Enabled
Encryption-3DES-AES : Enabled
Security
Contexts : 0
Carrier : Disabled
AnyConnect Premium
Peers : 2
AnyConnect
Essentials : Disabled
Other VPN Peers : 250
Total VPN Peers : 250
AnyConnect for
Mobile : Disabled
AnyConnect for Cisco
VPN Phone : Disabled
Advanced Endpoint
Assessment : Disabled
Shared License : Disabled
Total TLS Proxy
Sessions : 2
Botnet Traffic
Filter : Enabled
Cluster : Disabled
Serial Number:
9AK137KWDWB
Image type : Release
Key version : A
Configuration last
modified by enable_15 at 03:17:50.569 UTC Tue Jan 25 2022
Configure failover on ASAv-1 and ASAv-2 without the failover key command.
ASAv-1# configure terminal
ASAv-1(config)#
interface GigabitEthernet0/6 // DIRECT CABLE TO ASAv-2 G0/6
ASAv-1(config-if)# no shutdown
ASAv-1(config-if)#
failover lan unit primary
ASAv-1(config)#
failover lan interface FAILOVER GigabitEthernet0/6
INFO: Non-failover
interface config is cleared on GigabitEthernet0/6 and its sub-interfaces
ASAv-1(config)#
failover link FAILOVER GigabitEthernet0/6
ASAv-1(config)#
failover interface ip failover 10.0.0.1 255.255.255.252 standby 10.0.0.2
ASAv-1(config)#
failover
ASAv-1(config)# .
No
Active mate detected
Beginning configuration replication: Sending to
mate.
End Configuration Replication to mate
The ASAv pair started to sync and form the failover pair (or cluster) even without the failover key command.
ASAv-2# configure terminal
ASAv-2(config)#
interface GigabitEthernet0/6
ASAv-2(config-if)# no shutdown
ASAv-2(config-if)#
failover lan unit secondary
ASAv-2(config)#
failover lan interface FAILOVER GigabitEthernet0/6
INFO: Non-failover
interface config is cleared on GigabitEthernet0/6 and its sub-interfaces
ASAv-2(config)#
failover interface ip failover 10.0.0.1 255.255.255.252 standby 10.0.0.2
ASAv-2(config)#
failover
ASAv-2(config)# ..
Detected an Active mate
Beginning configuration replication from mate.
WARNING: Disabling
auto import may affect Smart Licensing
WARNING: Local user
database is empty and there are still 'aaa' commands for 'LOCAL'.
Creating trustpoint
"_SmartCallHome_ServerCA" and installing certificate...
Trustpoint CA
certificate accepted.
WARNING: Failover is
enabled but standby IP address is not configured for this interface.
WARNING: Failover is
enabled but standby IP address is not configured for this interface.
WARNING: Failover is
enabled but standby IP address is not configured for this interface.
WARNING: Trustpoint
_SmartCallHome_ServerCA is already authenticated.
WARNING: This
command will not take effect until interface 'inside' has been assigned an IPv4
address
End configuration
replication from mate.
The failover failed due to the monitored interfaces on ASAv-2. The only interface connected in ASAv-2 is a direct cable to ASAv-1 used for the failover interface (G0/6).
ASAv-1# ping
10.0.0.2
Type escape sequence
to abort.
Sending 5, 100-byte
ICMP Echos to 10.0.0.2, timeout is 2 seconds:
!!!!!
Success rate is 100
percent (5/5), round-trip min/avg/max = 1/8/20 ms
ASAv-1# show
failover
Failover On
Failover unit Primary
Failover LAN Interface: FAILOVER GigabitEthernet0/6
(up)
Reconnect timeout
0:00:00
Unit Poll frequency
1 seconds, holdtime 15 seconds
Interface Poll
frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces
3 of 61 maximum
MAC Address Move
Notification Interval not set
Version: Ours 9.8(1), Mate 9.8(1)
Serial Number: Ours
9A81V5LKN5F, Mate 9AK137KWDWB
Last Failover at:
03:20:54 UTC Jan 25 2022
This
host: Primary - Active
Active time: 52 (sec)
slot 0: empty
Interface outside
(200.1.1.1): Normal (Waiting)
Interface inside
(172.16.1.1): Normal (Waiting)
Interface dmz (10.1.1.1):
Normal (Waiting)
Other
host: Secondary - Failed
Active time: 0 (sec)
Interface outside (0.0.0.0):
No Link (Waiting)
Interface inside (0.0.0.0):
No Link (Waiting)
Interface dmz (0.0.0.0): No
Link (Waiting)
<OUTPUT
TRUNCATED>
ASAv-1# show
failover state
State Last Failure Reason Date/Time
This host -
Primary
Active None
Other host - Secondary
Failed
Ifc Failure
03:21:27 UTC Jan 25 2022
outside: No Link
inside: No Link
dmz: No Link
====Configuration
State===
Sync Done
====Communication
State===
Mac set
ASAv-1# show
failover history
==========================================================================
From State To State Reason
==========================================================================
03:02:35 UTC Jan 25
2022
Not Detected Disabled No Error
03:20:19 UTC Jan 25
2022
Disabled Negotiation Set by the config command
03:20:54 UTC Jan 25
2022
Negotiation Just Active No Active unit found
03:20:54 UTC Jan 25
2022
Just Active Active Drain No Active unit found
03:20:54 UTC Jan 25
2022
Active Drain Active Applying Config No Active unit found
03:20:54 UTC Jan 25
2022
Active Applying
Config Active Config Applied No Active unit found
03:20:54 UTC Jan 25
2022
Active Config
Applied Active No Active unit found
==========================================================================
I've temporarily removed monitoring on all interfaces using the no monitor-interface <INTERFACE> and added the prompt hostname priority command to distinguish between Primary and Secondary ASAv. The failover worked afterwards and Secondary ASAv status changed to Standby Ready.
ASAv-1(config)# no monitor-interface outside
ASAv-1(config)# no monitor-interface inside
ASAv-1(config)# no monitor-interface dmz
ASAv-1(config)#
prompt hostname priority
ASAv-1/pri(config)#
ASAv-1/sec#
Switching to Ok for reason Interface check.
ASAv-1/pri# show
failover
Failover On
Failover unit
Primary
Failover LAN
Interface: FAILOVER GigabitEthernet0/6 (up)
Reconnect timeout
0:00:00
Unit Poll frequency
1 seconds, holdtime 15 seconds
Interface Poll
frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces
0 of 61 maximum
MAC Address Move
Notification Interval not set
Version: Ours
9.8(1), Mate 9.8(1)
Serial Number: Ours
9A81V5LKN5F, Mate 9AK137KWDWB
Last Failover at:
03:20:54 UTC Jan 25 2022
This
host: Primary - Active
Active time: 277 (sec)
slot 0: empty
Interface outside
(200.1.1.1): Normal (Not-Monitored)
Interface inside
(172.16.1.1): Normal (Not-Monitored)
Interface dmz (10.1.1.1):
Normal (Not-Monitored)
Other
host: Secondary - Standby Ready
Active time: 0 (sec)
Interface outside (0.0.0.0):
No Link (Not-Monitored)
Interface inside (0.0.0.0):
No Link (Not-Monitored)
Interface dmz (0.0.0.0): No
Link (Not-Monitored)
<OUTPUT
TRUNCATED>
ASAv-1/pri# show
failover state
State Last Failure Reason Date/Time
This host -
Primary
Active None
Other host - Secondary
Standby Ready Ifc Failure 03:21:27 UTC Jan 25 2022
outside: No Link
inside: No Link
dmz: No Link
====Configuration
State===
Sync Done
====Communication
State===
Mac set
ASAv-1/pri# failover exec mate show run // VERIFY CONFIG IN ASAv-2
: Saved
:
: Serial Number:
9AK137KWDWB
: Hardware: ASAv, 2048 MB RAM, CPU Pentium II 1900 MHz
:
ASA Version 9.8(1)
!
hostname ASAv-1
enable password
$sha512$5000$5bps8k/6inHnfDTz/HO44A==$PjXAl3nW8pK5BfcT6tJYwA== pbkdf2
xlate per-session
deny tcp any4 any4
xlate per-session
deny tcp any4 any6
xlate per-session
deny tcp any6 any4
xlate per-session
deny tcp any6 any6
xlate per-session
deny udp any4 any4 eq domain
xlate per-session
deny udp any4 any6 eq domain
xlate per-session
deny udp any6 any4 eq domain
xlate per-session
deny udp any6 any6 eq domain
passwd
2KFQnbNIdI.2KYOU encrypted
names
!
interface
GigabitEthernet0/0
description ### TO R1 F1/0: INTERNET ###
nameif outside
security-level 0
ip address 200.1.1.1 255.255.255.0
!
interface
GigabitEthernet0/1
description ### TO IOU_SW01 E0/0: INSIDE ###
nameif inside
security-level 100
ip address 172.16.1.1 255.255.255.0
!
interface
GigabitEthernet0/2
description ### TO R2 DMZ SERVER ###
nameif dmz
security-level 50
<OUTPUT
TRUNCATED>
ASAv-1/pri# failover
exec mate show run failover
failover
failover lan unit
secondary
failover lan
interface FAILOVER GigabitEthernet0/6
failover link
FAILOVER GigabitEthernet0/6
failover interface
ip FAILOVER 10.0.0.1 255.255.255.252 standby 10.0.0.2
So I added the failover key command and confirmed it didn't "break" the ASA failover pair. The command was also immediately sync'd to ASAv-2.
ASAv-1/pri# configure terminal
ASAv-1/pri(config)# failover key cisco
ASAv-1/pri(config)#
end
ASAv-1/pri# show run
failover
failover
failover lan unit
primary
failover lan
interface FAILOVER GigabitEthernet0/6
failover key *****
failover link
FAILOVER GigabitEthernet0/6
failover interface
ip FAILOVER 10.0.0.1 255.255.255.252 standby 10.0.0.2
ASAv-1/pri# failover exec mate show run failover
failover
failover lan unit
secondary
failover lan
interface FAILOVER GigabitEthernet0/6
failover key *****
failover link
FAILOVER GigabitEthernet0/6
failover interface
ip FAILOVER 10.0.0.1 255.255.255.252 standby 10.0.0.2
ASAv-1/pri# sh
failover
Failover On
Failover unit
Primary
Failover LAN
Interface: FAILOVER GigabitEthernet0/6 (up)
Reconnect timeout
0:00:00
Unit Poll frequency
1 seconds, holdtime 15 seconds
Interface Poll
frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces
0 of 61 maximum
MAC Address Move
Notification Interval not set
Version: Ours
9.8(1), Mate 9.8(1)
Serial Number: Ours
9A81V5LKN5F, Mate 9AK137KWDWB
Last Failover at:
03:20:54 UTC Jan 25 2022
This
host: Primary - Active
Active time: 416 (sec)
slot 0: empty
Interface outside
(200.1.1.1): Normal (Not-Monitored)
Interface inside
(172.16.1.1): Normal (Not-Monitored)
Interface dmz (10.1.1.1):
Normal (Not-Monitored)
Other
host: Secondary - Standby Ready
Active time: 0 (sec)
Interface outside (0.0.0.0):
No Link (Not-Monitored)
Interface inside (0.0.0.0):
No Link (Not-Monitored)
Interface dmz (0.0.0.0): No
Link (Not-Monitored)
<OUTPUT
TRUNCATED>