AWS performance and scaling

Performance metrics (scan times)

Below are some estimates of how long a scan takes for various file types. The data was collected by uploading individual sample files to an S3 bucket that was in the same region as the scanner. The estimates assume that 512 MB of memory (the default) has been allocated to the File Storage Security scanner:

• EXE file under 100 MB: 4 seconds
• BIN file under 100 MB: 4 seconds
• JPEG file under 40 MB: 3 seconds
• MP4 file under 1600 MB: 3 seconds
• PDF file under 1500 MB: 22 seconds
• TXT file under 100 MB: 4 seconds
• ZIP file under 100 MB: 5 seconds

How are bursts in load handled? How do I estimate the scan time and how much concurrency is used when a burst of scanning occurs?

If a large number of scanning requests hit File Storage Security all at once, multiple instances of the ScannerLambda will be invoked to process the requests in parallel.

The exact scan time depends on the AWS concurrency settings of the Lambda functions, and well as how many other Lambdas you have running at the same time in your AWS account.

The Lambda concurrency setting number includes not only the Lambdas that File Storage Security uses, but also all the other Lambdas used in the same AWS account. You can leave this setting at its default (set by AWS) and make sure the Lambda concurrency in this AWS account is sufficient for the File Storage Security scan.

Below are some estimates of how long a scan takes and how much concurrency was used for various numbers of zipped 10 MB files. The data in the table was collected in a deployment where there was one storage stack and one scanner in the same region.

When the shared or reserved concurrency is lower than the requirement for the used concurrency, delays can be expected due to throttles.

In addition, there are three Lambda concurrencies invoked in the scan process, namely, the BucketListener, Scanner, and PostScanActionTag Lambda concurrencies. When a burst of traffic occurs in an architecture with one S3 bucket and one scanner, the BucketListener Lambda needs more concurrency.

How many files can be scanned concurrently?

The AWS Lambda service has a default setting of 1000 total allowable concurrent executions, and File Storage Security's ScannerLambda function follows this configuration. Thus, if two File Storage Security scanners are deployed under the same AWS account in the same region, the 1000 concurrent executions will be shared among the scanners (they won't each get 1000). Additionally, the concurrent executions will be further split with any other Lambdas deployed under the same AWS account in the same region.

The ScannerLambda receives scan messages by setting up an event source mapping to the SQS ScannerQueue. The maximum number of batches that an event source mapping can process simultaneously is 1000, and the batch size for the ScannerLambda is 1. In other words, ScannerLambda can poll at most 1000 scan messages from the SQS ScannerQueue simultaneously, even if there are still allowable concurrent executions. The polling latency determines the actual amount of scan messages that can be handled simultaneously. If the latency is 1 second, the maximum number of scan messages that can be handled in 1 second is bound to 1000. When the file uploading speed is higher than that amount, the scan messages are queued in the SQS ScannerQueue. If the scan messages are queued for more than one hour, the scanning process may time out. In this case, we suggest that you upload the files to different buckets or folders in a distributed way. Deploy one dedicated set of a Scanner Stack and a Storage stack for each bucket or folder. To scan a folder in a bucket, you can deploy the Storage Stack with setting ObjectFilterPrefix parameter to the folder's name with a trailing slash character (/).

File identification information is also scanned by the Trend Micro Global Smart Protection Server in the cloud. (See Architecture and flow for details.) This server is set up to handle very large loads from across the globe and will never cause a performance bottleneck.

Performance across multiple regions

File Storage Security's performance will be slower if the scanner stack is placed in a different continental region from its associated storage stacks. For example, if a scanner stack is in US N. Virginia (us-east-1), and its associated storage stacks are in Singapore (ap-southeast-1), then you'll experience a performance degradation. However, if the scanner and storage stacks are in the same general area, such as N. Virginia (us-east-1) and Ohio (us-east-2), then performance will remain optimal.

How can performance be improved?

If you would like better performance, you can:

• change your AWS settings to allow more concurrent executions. For details, see the AWS documentation: https://docs.aws.amazon.com/lambda/latest/dg/configuration-concurrency.html.
• configure provisioned concurrency on the Lambda alias TM-FSS-MANAGED. Warning: Do not configure routing on the Lambda alias TM-FSS-MANAGED. This may cause management problems.
• remove third-party Lambdas from the AWS account and region where the File Storage Security ScannerLambda function is deployed. This will allow you to devote execution power exclusively to the scanner.
• move your scanner stack into the same continental region as its associated storage stacks. This will allow you to avoid the slowdowns that may occur with inter-continental scanning.