GCP 云存储 (GCS) 新对象/更改电子邮件/文本、消息通知

Question

我们正在将一些 SQL 转储从 AWS RDS 存储到 GCP 云存储作为长期辅助备份。在过去的四天里，数据没有传输到 GCP，经过分析我们发现是因为夏令时的变化并解决了这个问题。

现在，将来我们希望从 GCP 收到一些通知，发送到电子邮件/短信 - 正面场景/寻呼机职责 - 负面场景（此时仅通过电子邮件）。发送电子邮件的积极场景是将新文件成功上传到 GCS 存储桶。但是由于 google 没有任何对电子邮件或文本的原生支持，请向我们建议一个解决方案。我配置了以下

已配置存储桶以通知对 pub sub 的更改 https://cloud.google.com/storage/docs/reporting-changes
将发布/订阅通知配置为拉取传递类型 https://cloud.google.com/storage/docs/pubsub-notifications

但现在我如何从 GCP 发送电子邮件/文本。我们是主要的 AWS 商店。我们可以使用 SES 或 SNS 或任何其他类型的通知从 PUB/SUB.

中获取数据吗

Answer 1

Can we use SES or SNS or any other type of notification to get the data out of PUB/SUB?

是的。我已经使用 Google Cloud Functions 和 Pub/Sub 触发器完成了此 (SES)。我假设 SNS 会一样简单。

但是，我发现设置 Google Cloud Functions 以使用 SMTP 客户端发送电子邮件要容易得多。我还使用 Twilio 发送短信。

要使用 AWS SNS 或 SES，您需要将 boto3 库与您的部署和 AWS 凭证打包在一起。您还可以使用 AWS REST 接口，这样就不需要外部库了。

对于电子邮件 SMTP 客户端，Google Cloud Functions 包含 smtplib，因此这是一条非常简单的路径。您只需要 SMTP 的电子邮件用户名和密码。为此，我使用 Gmail 或 Office 365 凭据。 20 行 python 代码，你就完成了。

[编辑]

使用 Google Cloud Functions 发送电子邮件时，请使用 SSL 进行传输。对于 Office 365 和 Gmail，此端口 587。不要尝试使用不使用 SSL 的端口 25，因为几乎每个人都会阻止端口 25。

Answer 2

我们可以配置GCS bucket 来直接触发云函数，用于对GCS bucket 进行以下更改。
1. 完成 - 创建
2. 删除
3. 存档
4. 元数据更新

教程：https://cloud.google.com/functions/docs/tutorials/storage

云函数 python 3.7 使用 Amazon SES SMTP 接口触发电子邮件。

import smtplib
import email.utils
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText

def email_notify(event, context):
    """Triggered by a change to a Cloud Storage bucket.
    Args:
    event (dict): Event payload.
    context (google.cloud.functions.Context): Metadata for the event.
    """
    file = event
    print(f"Processing file: {file['name']}.")
    BUCKET = file['bucket']
    FILE = file['name']
    CREATED = file['timeCreated']
    UPDATED = file['updated']
    # Replace sender@example.com with your "From" address.
    # This address must be verified.
    SENDER = 'sender-email-address'  
    SENDERNAME = 'no-reply'

    # Replace recipient@example.com with a "To" address. If your account
    # is still in the sandbox, this address must be verified.
    RECIPIENT  = 'recepient-email-address'

    # Replace smtp_username with your Amazon SES SMTP user name.
    USERNAME_SMTP = "SMTP-USERNAME"

    # Replace smtp_password with your Amazon SES SMTP password.
    PASSWORD_SMTP = "SMTP-PASSWORD"

    # (Optional) the name of a configuration set to use for this message.
    # If you comment out this line, you also need to remove or comment out
    # the "X-SES-CONFIGURATION-SET:" header below.
    # CONFIGURATION_SET = "ConfigSet"

    # If you're using Amazon SES in an AWS Region other than US West (Oregon),
    # replace email-smtp.us-west-2.amazonaws.com with the Amazon SES SMTP
    # endpoint in the appropriate region.
    HOST = "email-smtp.us-west-2.amazonaws.com"
    PORT = 587

    # The subject line of the email.
    SUBJECT = 'Successfull upload of file {} to GCS bucket {}'.format(FILE,BUCKET)

    # The email body for recipients with non-HTML email clients.
    BODY_TEXT = ("File upload to GCS bucket\r\n"
                 "Bucket-Name: {}\r\n"
                 "File-Name: {}\r\n"
                 "File-Create-Date: {}\r\n"
                 "File-Update-Date: {}\r\n"
                ).format(BUCKET,FILE,CREATED,UPDATED)

    # Create message container - the correct MIME type is multipart/alternative.
    msg = MIMEMultipart('alternative')
    msg['Subject'] = SUBJECT
    msg['From'] = email.utils.formataddr((SENDERNAME, SENDER))
    msg['To'] = RECIPIENT
    # Comment or delete the next line if you are not using a configuration set
    # msg.add_header('X-SES-CONFIGURATION-SET',CONFIGURATION_SET)

    # Record the MIME types of both parts - text/plain and text/html.
    part1 = MIMEText(BODY_TEXT, 'plain')
    # part2 = MIMEText(BODY_HTML, 'html')

    # Attach parts into message container.
    # According to RFC 2046, the last part of a multipart message, in this case
    # the HTML message, is best and preferred.
    msg.attach(part1)
    # msg.attach(part2)

    # Try to send the message.
    try:
        server = smtplib.SMTP(HOST, PORT)
        server.ehlo()
        server.starttls()
        #stmplib docs recommend calling ehlo() before & after starttls()
        server.ehlo()
        server.login(USERNAME_SMTP, PASSWORD_SMTP)
        server.sendmail(SENDER, RECIPIENT, msg.as_string())
        server.close()
    # Display an error message if something goes wrong.
    except Exception as e:
        print ("Error: ", e)
    else:
        print ("Email sent!")

GCP 云存储 (GCS) 新对象/更改电子邮件/文本、消息通知

GCP Cloud Storage(GCS) new object / changes notifications to email / text ,message

amazon-sns

amazon-ses

google-cloud-storage

google-cloud-platform

google-cloud-pubsub