Package org.graylog.plugins.map.config
Class S3GeoIpFileService
java.lang.Object
org.graylog.plugins.map.config.S3GeoIpFileService
Service for pulling Geo Location Processor ASN and city database files from an S3 bucket and storing them on disk.
The files will initially be downloaded to a temporary location on disk, then they will be validated by the
GeoIpResolverConfigValidator
, and after successful validation they will
be moved to the active location so that the Geo Location Processor can read them. The on-disk directory location
for downloaded files is S3_DOWNLOAD_LOCATION in GeoIpProcessorConfig
. The file names are hardcoded to ensure
that the proper files are always left active.
This service is called from two places:
- GeoIpResolverConfigValidator
will download new files when the Geo
Location Processor configuration is changed and the new configuration has different S3 objects than the old.
- GeoIpDbFileChangeMonitorService
will check to see if new files need to be
downloaded each time the service runs based on the lastModified times of the S3 objects.
This class relies on the DefaultCredentialsProvider and not any settings that may be configured in the
Graylog AWS plugin configuration. See https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/credentials.html#credentials-chain
for how to configure your environment so that the default provider retrieves credentials properly.-
Field Summary
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionvoid
Delete the temporary files if they exist and reset their last modified timesvoid
Downloads the Geo Processor city and ASN database files to a temporary location so that they can be validatedboolean
Checks to see if either the database files need to be pulled down from S3Get the path to where the active ASN database file will be stored on disk.Get the path to where the active city database file will be stored on disk.Get the path to where the temporary ASN database file will be stored on diskGet the path to where the temporary city database file will be stored on diskvoid
Once the database files have been downloaded from S3 and then validated, move them to a fixed location for the Geo Location processor to read and update the last modified variables.boolean
-
Field Details
-
S3_BUCKET_PREFIX
- See Also:
-
ACTIVE_ASN_FILE
- See Also:
-
ACTIVE_CITY_FILE
- See Also:
-
TEMP_ASN_FILE
- See Also:
-
TEMP_CITY_FILE
- See Also:
-
NULL_S3_CLIENT_MESSAGE
- See Also:
-
-
Constructor Details
-
S3GeoIpFileService
-
-
Method Details
-
downloadFilesToTempLocation
Downloads the Geo Processor city and ASN database files to a temporary location so that they can be validated- Parameters:
config
- current Geo Location Processor configuration- Throws:
S3DownloadException
- if the files fail to be downloaded
-
fileRefreshRequired
Checks to see if either the database files need to be pulled down from S3- Parameters:
config
- current Geo Location Processor configuration- Returns:
- true if the files in S3 have been modified since they were last synced
-
moveTempFilesToActive
Once the database files have been downloaded from S3 and then validated, move them to a fixed location for the Geo Location processor to read and update the last modified variables.- Throws:
IOException
- if the files fail to be moved to the active location
-
getTempAsnFile
Get the path to where the temporary ASN database file will be stored on disk- Returns:
- temporary ASN database file path
-
getTempCityFile
Get the path to where the temporary city database file will be stored on disk- Returns:
- temporary city database file path
-
getActiveAsnFile
Get the path to where the active ASN database file will be stored on disk. The file here will always be used by the Geo Location Processor if the Use S3 config option is enabled.- Returns:
- active ASN database file path
-
getActiveCityFile
Get the path to where the active city database file will be stored on disk. The file here will always be used by the Geo Location Processor if the Use S3 config option is enabled.- Returns:
- active city database file path
-
cleanupTempFiles
public void cleanupTempFiles()Delete the temporary files if they exist and reset their last modified times -
s3ClientIsNull
public boolean s3ClientIsNull()
-