Diego 安装 - 使用 spiff merge 生成清单时出错
Diego Installation - Error when generating manifest with spiff merge
当 "generating manifest" 由 运行:
spiff merge input/config-from-cf.yml input/config-from-cf-internal.yml input/cf-deployment.yml > output/config-from-cf.yml
我收到以下错误。我做错了什么？看起来节点包含在最后一个文件中。他们为什么不解决？
error generating manifest: unresolved nodes:
(( properties.cc.internal_api_password )) in dynaml config_from_cf.cc.internal_api_password ()
(( properties.cc.srv_api_uri )) in dynaml config_from_cf.cc.srv_api_uri ()
(( properties.cc.staging_upload_password )) in dynaml config_from_cf.cc.staging_upload_password ()
(( properties.cc.staging_upload_user )) in dynaml config_from_cf.cc.staging_upload_user ()
(( name )) in dynaml config_from_cf.cf_deployment_name ()
(( properties.consul.agent.servers.lan )) in dynaml config_from_cf.consul.lan_servers ()
(( properties.consul.agent.log_level )) in dynaml config_from_cf.consul.log_level ()
(( properties.etcd.machines )) in dynaml config_from_cf.etcd.machines ()
(( properties.loggregator_endpoint.shared_secret )) in dynaml config_from_cf.loggregator_endpoint.shared_secret ()
(( properties.nats.machines )) in dynaml config_from_cf.nats.machines ()
(( properties.nats.password )) in dynaml config_from_cf.nats.password ()
(( properties.nats.user )) in dynaml config_from_cf.nats.user ()
(( properties.system_domain )) in dynaml config_from_cf.system_domain ()
(( merge )) in input/config-from-cf-internal.yml name (name)
(( merge )) in input/config-from-cf-internal.yml properties.etcd.machines (properties.etcd.machines)
(( merge )) in input/config-from-cf-internal.yml properties.loggregator_endpoint.shared_secret (properties.loggregator_endpoint.shared_secret)
(( merge )) in input/config-from-cf-internal.yml properties.nats.password (properties.nats.password)
(( merge )) in input/config-from-cf-internal.yml properties.nats.user (properties.nats.user)
(( merge )) in input/config-from-cf-internal.yml properties.nats.machines (properties.nats.machines)
(( merge )) in input/config-from-cf-internal.yml properties.system_domain (properties.system_domain)
(( merge )) in input/config-from-cf-internal.yml properties.cc.internal_api_password (properties.cc.internal_api_password)
(( merge )) in input/config-from-cf-internal.yml properties.cc.srv_api_uri (properties.cc.srv_api_uri)
(( merge )) in input/config-from-cf-internal.yml properties.cc.staging_upload_password (properties.cc.staging_upload_password)
(( merge )) in input/config-from-cf-internal.yml properties.cc.staging_upload_user (properties.cc.staging_upload_user)
(( merge )) in input/config-from-cf-internal.yml properties.consul.agent.log_level (properties.consul.agent.log_level)
(( merge )) in input/config-from-cf-internal.yml properties.consul.agent.servers.lan (properties.consul.agent.servers.lan)
我用于 spiff 合并的文件的内容。
config-from-cf.yml:
config_from_cf:
cf_deployment_name: (( merge ))
cc:
internal_api_password: (( merge ))
srv_api_uri: (( merge ))
staging_upload_user: (( merge ))
staging_upload_password: (( merge ))
consul:
log_level: (( merge ))
lan_servers: (( merge ))
ca_cert: (( merge ))
agent_cert: (( merge ))
agent_key: (( merge ))
encrypt_keys: (( merge ))
require_ssl: (( merge ))
server_cert: (( merge ))
server_key: (( merge ))
etcd:
machines: (( merge ))
loggregator_endpoint:
shared_secret: (( merge ))
nats:
user: (( merge ))
password: (( merge ))
port: (( merge ))
machines: (( merge ))
system_domain: (( merge ))
uaa:
clients:
ssh-proxy:
secret: (( merge ))
url: (( merge || nil ))
config-from-cf-internal.yml:
config_from_cf:
cf_deployment_name: (( name ))
cc:
internal_api_password: (( properties.cc.internal_api_password ))
srv_api_uri: (( properties.cc.srv_api_uri ))
staging_upload_user: (( properties.cc.staging_upload_user ))
staging_upload_password: (( properties.cc.staging_upload_password ))
consul:
log_level: (( properties.consul.agent.log_level ))
lan_servers: (( properties.consul.agent.servers.lan ))
ca_cert: (( properties.consul.ca_cert ))
agent_cert: (( properties.consul.agent_cert ))
agent_key: (( properties.consul.agent_key ))
encrypt_keys: (( properties.consul.encrypt_keys ))
require_ssl: (( properties.consul.require_ssl ))
server_cert: (( properties.consul.server_cert ))
server_key: (( properties.consul.server_key ))
etcd:
machines: (( properties.etcd.machines ))
loggregator_endpoint:
shared_secret: (( properties.loggregator_endpoint.shared_secret ))
nats:
user: (( properties.nats.user ))
password: (( properties.nats.password ))
port: 4222
machines: (( properties.nats.machines ))
system_domain: (( properties.system_domain ))
uaa:
url: (( properties.uaa.url ))
clients:
ssh-proxy:
secret: (( properties.uaa.clients.ssh-proxy.secret ))
# The keys below should not be included in the final stub
name: (( merge ))
properties:
cc:
internal_api_password: (( merge ))
srv_api_uri: (( merge ))
staging_upload_user: (( merge ))
staging_upload_password: (( merge ))
consul:
agent:
log_level: (( merge ))
servers:
lan: (( merge ))
ca_cert:
agent_cert:
agent_key:
encrypt_keys:
require_ssl:
server_cert:
server_key:
etcd:
machines: (( merge ))
loggregator_endpoint:
shared_secret: (( merge ))
nats:
user: (( merge ))
password: (( merge ))
machines: (( merge ))
system_domain: (( merge ))
uaa:
clients:
ssh-proxy:
secret: (( merge || nil ))
url: (( merge || nil ))
cf-deployment.yml
#######################################################################
####### VARIABLES
#######################################################################
<%
director_uuid = 'XXX'
static_ip = 'XXX'
root_domain = "#{static_ip}.XXX"
cf_deployment_name = 'XXX'
cf_release_name = 'XXX'
cf_release_version = '0.0.1'
protocol = 'http'
flavorSmall = 'c1.medium'
flavorMedium = 'c1.large'
flavorLarge = 'c1.xlarge'
flavorXlarge = 'c1.xxlarge'
networkInternal = 'XXX'
networkInternalId = 'XXX'
networkInternalCIDR = 'XXX'
networkInternalGateway = 'XXX'
networkInternalSubnetId = 'XXX'
networkInternalDNS = ''
networkExternal = 'XXX'
networkExternalId = 'XXX'
networkExternalSubnetId = 'XXX'
networkExternalCIDR = 'XXX'
networkExternalGateway = 'XXX'
networkExternalDNS = 'XXX, XXX'
networkPublic = 'public'
networkPublicId = 'XXX'
IpInternalPostgres = 'XXX'
IpInternalRouter = 'XXX'
IpInternalNats = 'XXX'
IpInternalEtcd = 'XXX'
IpInternalNfs = 'XXX'
IpInternalConsul1 = 'XXX'
IpInternalConsul2 = 'XXX'
IpInternalConsul3 = 'XXX'
boshStemcell = 'bosh-openstack-kvm-ubuntu-trusty-go_agent-raw'
jobs_doppler_shared_secret = 'XXX'
jobs_uaa_admin_client_secret = 'XXX'
jobs_uaa_batch_password = 'XXX'
jobs_uaa_cc_client_secret = 'XXX'
jobs_uaa_scim_cc_client_secret = 'XXX'
properties_cc_db_encryption_key = 'XXX'
properties_cc_bulk_api_password = 'XXX'
properties_cc_internal_api_password = 'XXX'
properties_cc_staging_upload_password = 'XXX'
uaa_clients_cc_service_dashboards_password = 'XXX'
uaa_clients_cloud_controller_username_lookup_password = 'XXX'
uaa_clients_cc_routing_password = 'XXX'
uaa_clients_gorouter_password = 'XXX'
uaa_clients_login_password = 'XXX'
uaa_clients_servicesmgmt_password = 'XXX'
nats_password = 'XXX'
ccdb_roles_ccadmin_password = 'XXX'
ccdb_roles_uaaadmin_password = 'XXX'
loggregator_endpoint_shared_secret = 'XXX'
metron_endpoint_shared_secret = 'XXX'
%>
---
#######################################################################
####### NETWORK AND MISCELLANEOUS
#######################################################################
name: <%= cf_deployment_name %>
director_uuid: <%= director_uuid %>
releases:
- {name: <%= cf_release_name %>, version: <%= cf_release_version %>}
networks:
- name: <%= networkInternal %>
type: manual
subnets:
- range: <%= networkInternalCIDR %>
gateway: <%= networkInternalGateway %>
dns: [<%= networkInternalDNS %>]
reserved: ["XXX - XXX"]
static: ["XXX - XXX"]
cloud_properties:
net_id: <%= networkInternalId %>
security_groups:
- default
- XXX
- XXX
subnet: networkInternalSubnetId
- name: <%= networkExternal %>
type: manual
subnets:
- range: <%= networkExternalCIDR %>
gateway: <%= networkExternalGateway %>
dns: [<%= networkExternalDNS %>]
reserved: ["XXX - XXX"]
static: ["XXX - XXX"]
cloud_properties:
net_id: <%= networkExternalId %>
security_groups:
- default
- XXX
- XXX
- XXX
subnet: <%= networkExternalSubnetId %>
- name: <%= networkPublic %>
type: vip
cloud_properties:
subnet: []
resource_pools:
- name: common_z1
network: <%= networkInternal %>
stemcell:
name: <%= boshStemcell %>
version: latest
cloud_properties:
instance_type: <%= flavorSmall %>
- name: large_z1
network: <%= networkInternal %>
stemcell:
name: <%= boshStemcell %>
version: latest
cloud_properties:
instance_type: <%= flavorMedium %>
compilation:
workers: 2
network: <%= networkInternal %>
reuse_compilation_vms: true
cloud_properties:
instance_type: <%= flavorMedium %>
update:
canaries: 1
max_in_flight: 32
serial: false
canary_watch_time: 30000-600000
update_watch_time: 30000-600000
#######################################################################
####### JOBS
#######################################################################
jobs:
- name: nats_z1
instances: 1
resource_pool: common_z1
templates:
- {name: nats, release: <%= cf_release_name %>}
- {name: nats_stream_forwarder, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
static_ips: [<%= IpInternalNats %>]
- name: etcd_z1
instances: 1
resource_pool: common_z1
persistent_disk: 102400
templates:
- {name: etcd, release: <%= cf_release_name %>}
- {name: etcd_metrics_server, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
static_ips: [<%= IpInternalEtcd %>]
properties:
etcd_metrics_server:
nats:
machines: [<%= IpInternalNats %>]
password: <%= nats_password %>
username: nats
- name: nfs_z1
instances: 1
persistent_disk: 102400
resource_pool: common_z1
templates:
- {name: debian_nfs_server, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
static_ips: [<%= IpInternalNfs %>]
- name: postgres_z1
instances: 1
persistent_disk: 1024
resource_pool: common_z1
templates:
- {name: postgres, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
static_ips: [<%= IpInternalPostgres %>]
update:
serial: true
- name: api_z1
instances: 1
resource_pool: common_z1
templates:
- {name: cloud_controller_ng, release: <%= cf_release_name %>}
- {name: cloud_controller_worker, release: <%= cf_release_name %>}
- {name: cloud_controller_clock, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
- {name: nfs_mounter, release: <%= cf_release_name %>}
- {name: route_registrar, release: <%= cf_release_name %>}
- {name: consul_agent, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
properties:
nfs_server:
address: <%= IpInternalNfs %>
allow_from_entries: [<%= networkInternalCIDR %>]
route_registrar:
routes:
- name: api
port: 9022
uris:
- "api.<%= root_domain %>"
- name: ha_proxy_z1
instances: 1
resource_pool: common_z1
templates:
- {name: haproxy, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
networks:
- name: <%= networkPublic %>
static_ips: [<%= static_ip %>]
- name: <%= networkExternal %>
default: [gateway, dns]
properties:
ha_proxy:
ssl_pem: |
-----BEGIN CERTIFICATE-----
XXX
-----END CERTIFICATE-----
-----BEGIN RSA PRIVATE KEY-----
XXX
-----END RSA PRIVATE KEY-----
router:
servers:
z1: [<%= IpInternalRouter %>]
- name: hm9000_z1
instances: 1
resource_pool: common_z1
templates:
- {name: hm9000, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
- {name: route_registrar, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
properties:
route_registrar:
routes:
- name: hm9000
port: 5155
uris:
- "hm9000.<%= root_domain %>"
- name: doppler_z1
instances: 1
resource_pool: common_z1
templates:
- {name: doppler, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
properties:
doppler: {zone: z1}
doppler_endpoint:
shared_secret: <%= jobs_doppler_shared_secret %>
- name: loggregator_trafficcontroller_z1
instances: 1
resource_pool: common_z1
templates:
- {name: loggregator_trafficcontroller, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
- {name: route_registrar, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
properties:
traffic_controller: {zone: z1}
route_registrar:
routes:
- name: doppler
port: 8081
uris:
- "doppler.<%= root_domain %>"
- name: loggregator
port: 8080
uris:
- "loggregator.<%= root_domain %>"
- name: uaa_z1
instances: 1
resource_pool: common_z1
templates:
- {name: uaa, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
- {name: route_registrar, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
properties:
login:
catalina_opts: -Xmx768m #-XX:MaxPermSize=256m
route_registrar:
routes:
- name: uaa
port: 8080
uris:
- "uaa.<%= root_domain %>"
- "*.uaa.<%= root_domain %>"
- "login.<%= root_domain %>"
- "*.login.<%= root_domain %>"
uaa:
admin:
client_secret: <%= jobs_uaa_admin_client_secret %>
batch:
password: <%= jobs_uaa_batch_password %>
username: batch_user
cc:
client_secret: <%= jobs_uaa_cc_client_secret %>
scim:
userids_enabled: true
users:
- admin|<%= jobs_uaa_scim_cc_client_secret %>|scim.write,scim.read,openid,cloud_controller.admin,doppler.firehose
uaadb:
address: <%= IpInternalPostgres %>
databases:
- {name: uaadb, tag: uaa}
db_scheme: postgresql
port: 5524
roles:
- {name: uaaadmin, password: <%= ccdb_roles_uaaadmin_password %>, tag: admin}
- name: router_z1
instances: 1
resource_pool: common_z1
templates:
- {name: gorouter, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
static_ips: [<%= IpInternalRouter %>]
properties:
dropsonde: {enabled: true}
- name: runner_z1
instances: 1
resource_pool: common_z1
templates:
- {name: dea_next, release: <%= cf_release_name %>}
- {name: dea_logging_agent, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
properties:
dea_next: {zone: z1}
- name: stats_z1
instances: 1
resource_pool: common_z1
templates:
- {name: collector, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
properties:
collector: {deployment_name: <%= cf_deployment_name %>}
- name: consul_z1
instances: 1
resource_pool: common_z1
networks:
- name: <%= networkInternal %>
static_ips: [<%= IpInternalConsul1 %>]
templates:
- {name: consul_agent, release: <%= cf_release_name %>}
# - name: consul_z2
# instances: 0
# networks:
# - name: <%= networkExternal %>
# static_ips: []
#######################################################################
####### Properties
#######################################################################
properties:
networks: {apps: <%= networkInternal %>}
app_domains: [<%= root_domain %>]
cc:
allow_app_ssh_access: false
bulk_api_password: <%= properties_cc_bulk_api_password %>
db_encryption_key: <%= properties_cc_db_encryption_key %>
default_running_security_groups: [public_networks, dns]
default_staging_security_groups: [public_networks, dns]
install_buildpacks:
- {name: java_buildpack, package: buildpack_java}
- {name: ruby_buildpack, package: buildpack_ruby}
- {name: nodejs_buildpack, package: buildpack_nodejs}
- {name: go_buildpack, package: buildpack_go}
- {name: python_buildpack, package: buildpack_python}
- {name: php_buildpack, package: buildpack_php}
- {name: staticfile_buildpack, package: buildpack_staticfile}
- {name: binary_buildpack, package: buildpack_binary}
internal_api_password: <%= properties_cc_internal_api_password %>
quota_definitions:
default:
memory_limit: 4000 #MB
non_basic_services_allowed: false
total_routes: 20
total_services: -1
iron_quota:
memory_limit: 8000 #MB
non_basic_services_allowed: false
total_routes: 100
total_services: 40
bronze_quota:
memory_limit: 16000 #MB
non_basic_services_allowed: false
total_routes: 300
total_services: 300
silver_quota:
memory_limit: 32000 #MB
non_basic_services_allowed: false
total_routes: 1000
total_services: 1000
gold_quota:
memory_limit: 64000 #MB
non_basic_services_allowed: true
total_routes: 5000
total_services: 2000
trial_db_allowed: true
security_group_definitions:
- name: public_networks
rules:
- {destination: XXX-XXX, protocol: all}
- {destination: XXX-XXX, protocol: all}
- {destination: XXX-XXX, protocol: all}
- {destination: XXX-XXX, protocol: all}
- {destination: XXX-XXX, protocol: all}
- name: dns
rules:
- {destination: XXX, ports: '53', protocol: tcp}
- {destination: XXX, ports: '53', protocol: udp}
srv_api_uri: http://api.<%= root_domain %>
staging_upload_password: <%= properties_cc_staging_upload_password %>
staging_upload_user: staging_upload_user
ccdb:
address: <%= IpInternalPostgres %>
databases:
- {name: ccdb, tag: cc}
db_scheme: postgres
port: 5524
roles:
- {name: ccadmin, password: <%= ccdb_roles_ccadmin_password %>, tag: admin}
databases:
databases:
- {name: ccdb, tag: cc, citext: true}
- {name: uaadb, tag: uaa, citext: true}
port: 5524
roles:
- {name: ccadmin, password: <%= ccdb_roles_ccadmin_password %>, tag: admin}
- {name: uaaadmin, password: <%= ccdb_roles_uaaadmin_password %>, tag: admin}
dea_next:
advertise_interval_in_seconds: 5
heartbeat_interval_in_seconds: 10
memory_mb: 33996
description: XXX
domain: <%= root_domain %>
etcd:
machines: [<%= IpInternalEtcd %>]
hm9000:
url: http://hm9000.<%= root_domain %>
logger_endpoint:
port: 4443
loggregator_endpoint:
shared_secret: <%= loggregator_endpoint_shared_secret %>
login:
protocol: http
metron_agent:
zone: z1
deployment: minimal-aws
metron_endpoint:
shared_secret: <%= metron_endpoint_shared_secret %>
nats:
machines: [<%= IpInternalNats %>]
password: <%= nats_password %>
port: 4222
user: nats
nfs_server:
address: <%= IpInternalNfs %>
allow_from_entries: [<%= networkInternalCIDR %>]
ssl:
skip_cert_verify: true
system_domain: <%= root_domain %>
system_domain_organization: default_organization
uaa:
clients:
cc-service-dashboards:
authorities: clients.read,clients.write,clients.admin
authorized-grant-types: client_credentials
scope: openid,cloud_controller_service_permissions.read
secret: <%= uaa_clients_cc_service_dashboards_password %>
cloud_controller_username_lookup:
authorities: scim.userids
authorized-grant-types: client_credentials
secret: <%= uaa_clients_cloud_controller_username_lookup_password %>
cc_routing:
authorities: routing.router_groups.read
secret: <%= uaa_clients_cc_routing_password %>
authorized-grant-types: client_credentials
gorouter:
authorities: clients.read,clients.write,clients.admin,routing.routes.write,routing.routes.read
authorized-grant-types: client_credentials,refresh_token
scope: openid,cloud_controller_service_permissions.read
secret: <%= uaa_clients_gorouter_password %>
doppler:
authorities: uaa.resource
secret: <%= jobs_doppler_shared_secret %>
login:
authorities: oauth.login,scim.write,clients.read,notifications.write,critical_notifications.write,emails.write,scim.userids,password.write
authorized-grant-types: authorization_code,client_credentials,refresh_token
redirect-uri: http://login.<%= root_domain %>
scope: openid,oauth.approvals
secret: <%= uaa_clients_login_password %>
servicesmgmt:
authorities: uaa.resource,oauth.service,clients.read,clients.write,clients.secret
authorized-grant-types: authorization_code,client_credentials,password,implicit
autoapprove: true
redirect-uri: http://servicesmgmt.<%= root_domain %>/auth/cloudfoundry/callback
scope: openid,cloud_controller.read,cloud_controller.write
secret: <%= uaa_clients_servicesmgmt_password %>
jwt:
signing_key: |
-----BEGIN RSA PRIVATE KEY-----
XXX
-----END RSA PRIVATE KEY-----
verification_key: |
-----BEGIN PUBLIC KEY-----
XXX
-----END PUBLIC KEY-----
no_ssl: true
url: http://uaa.<%= root_domain %>
consul:
encrypt_keys:
- XXX
require_ssl: true
ca_cert: |
-----BEGIN CERTIFICATE-----
XXX
-----END CERTIFICATE-----
server_cert: |
-----BEGIN CERTIFICATE-----
XXX
-----END CERTIFICATE-----
server_key: |
-----BEGIN RSA PRIVATE KEY-----
XXX
-----END RSA PRIVATE KEY-----
agent_cert: |
-----BEGIN CERTIFICATE-----
XXX
-----END CERTIFICATE-----
agent_key: |
-----BEGIN RSA PRIVATE KEY-----
XXX
-----END RSA PRIVATE KEY-----
YAML 与顶部的所有变量混淆，然后是 ---。它将其视为文档分隔符，因此基本上只读取 --- 上方的内容。您可以阅读有关 YAML 文档分隔符的更多信息 here。
如果删除第一个三重破折号以上的所有内容，您的 spiff 合并会更进一步，但是由于缺少 consul 属性，您仍然会失败。所以你至少需要添加这些。
即便如此，您也会遇到问题，因为您打算将此工具与完全解析的 CF 部署清单一起用作第三个参数，而不是仍然包含 ERB 的东西（您的 input/cf-deployment.yml 不会转移到 output/config-from-cf.yml。如果您已经对仍然有 ERB 的 CF 清单完成 bosh deploy，您可以通过执行 bosh download manifest XXX input/resolved-cf-deployment.yml 然后 运行:
spiff merge input/config-from-cf.yml \
input/config-from-cf-internal.yml \
input/resolved-cf-deployment.yml \
> output/config-from-cf.yml
当 "generating manifest" 由 运行:
spiff merge input/config-from-cf.yml input/config-from-cf-internal.yml input/cf-deployment.yml > output/config-from-cf.yml
我收到以下错误。我做错了什么？看起来节点包含在最后一个文件中。他们为什么不解决？
error generating manifest: unresolved nodes:
(( properties.cc.internal_api_password )) in dynaml config_from_cf.cc.internal_api_password ()
(( properties.cc.srv_api_uri )) in dynaml config_from_cf.cc.srv_api_uri ()
(( properties.cc.staging_upload_password )) in dynaml config_from_cf.cc.staging_upload_password ()
(( properties.cc.staging_upload_user )) in dynaml config_from_cf.cc.staging_upload_user ()
(( name )) in dynaml config_from_cf.cf_deployment_name ()
(( properties.consul.agent.servers.lan )) in dynaml config_from_cf.consul.lan_servers ()
(( properties.consul.agent.log_level )) in dynaml config_from_cf.consul.log_level ()
(( properties.etcd.machines )) in dynaml config_from_cf.etcd.machines ()
(( properties.loggregator_endpoint.shared_secret )) in dynaml config_from_cf.loggregator_endpoint.shared_secret ()
(( properties.nats.machines )) in dynaml config_from_cf.nats.machines ()
(( properties.nats.password )) in dynaml config_from_cf.nats.password ()
(( properties.nats.user )) in dynaml config_from_cf.nats.user ()
(( properties.system_domain )) in dynaml config_from_cf.system_domain ()
(( merge )) in input/config-from-cf-internal.yml name (name)
(( merge )) in input/config-from-cf-internal.yml properties.etcd.machines (properties.etcd.machines)
(( merge )) in input/config-from-cf-internal.yml properties.loggregator_endpoint.shared_secret (properties.loggregator_endpoint.shared_secret)
(( merge )) in input/config-from-cf-internal.yml properties.nats.password (properties.nats.password)
(( merge )) in input/config-from-cf-internal.yml properties.nats.user (properties.nats.user)
(( merge )) in input/config-from-cf-internal.yml properties.nats.machines (properties.nats.machines)
(( merge )) in input/config-from-cf-internal.yml properties.system_domain (properties.system_domain)
(( merge )) in input/config-from-cf-internal.yml properties.cc.internal_api_password (properties.cc.internal_api_password)
(( merge )) in input/config-from-cf-internal.yml properties.cc.srv_api_uri (properties.cc.srv_api_uri)
(( merge )) in input/config-from-cf-internal.yml properties.cc.staging_upload_password (properties.cc.staging_upload_password)
(( merge )) in input/config-from-cf-internal.yml properties.cc.staging_upload_user (properties.cc.staging_upload_user)
(( merge )) in input/config-from-cf-internal.yml properties.consul.agent.log_level (properties.consul.agent.log_level)
(( merge )) in input/config-from-cf-internal.yml properties.consul.agent.servers.lan (properties.consul.agent.servers.lan)
我用于 spiff 合并的文件的内容。
config-from-cf.yml:
config_from_cf:
cf_deployment_name: (( merge ))
cc:
internal_api_password: (( merge ))
srv_api_uri: (( merge ))
staging_upload_user: (( merge ))
staging_upload_password: (( merge ))
consul:
log_level: (( merge ))
lan_servers: (( merge ))
ca_cert: (( merge ))
agent_cert: (( merge ))
agent_key: (( merge ))
encrypt_keys: (( merge ))
require_ssl: (( merge ))
server_cert: (( merge ))
server_key: (( merge ))
etcd:
machines: (( merge ))
loggregator_endpoint:
shared_secret: (( merge ))
nats:
user: (( merge ))
password: (( merge ))
port: (( merge ))
machines: (( merge ))
system_domain: (( merge ))
uaa:
clients:
ssh-proxy:
secret: (( merge ))
url: (( merge || nil ))
config-from-cf-internal.yml:
config_from_cf:
cf_deployment_name: (( name ))
cc:
internal_api_password: (( properties.cc.internal_api_password ))
srv_api_uri: (( properties.cc.srv_api_uri ))
staging_upload_user: (( properties.cc.staging_upload_user ))
staging_upload_password: (( properties.cc.staging_upload_password ))
consul:
log_level: (( properties.consul.agent.log_level ))
lan_servers: (( properties.consul.agent.servers.lan ))
ca_cert: (( properties.consul.ca_cert ))
agent_cert: (( properties.consul.agent_cert ))
agent_key: (( properties.consul.agent_key ))
encrypt_keys: (( properties.consul.encrypt_keys ))
require_ssl: (( properties.consul.require_ssl ))
server_cert: (( properties.consul.server_cert ))
server_key: (( properties.consul.server_key ))
etcd:
machines: (( properties.etcd.machines ))
loggregator_endpoint:
shared_secret: (( properties.loggregator_endpoint.shared_secret ))
nats:
user: (( properties.nats.user ))
password: (( properties.nats.password ))
port: 4222
machines: (( properties.nats.machines ))
system_domain: (( properties.system_domain ))
uaa:
url: (( properties.uaa.url ))
clients:
ssh-proxy:
secret: (( properties.uaa.clients.ssh-proxy.secret ))
# The keys below should not be included in the final stub
name: (( merge ))
properties:
cc:
internal_api_password: (( merge ))
srv_api_uri: (( merge ))
staging_upload_user: (( merge ))
staging_upload_password: (( merge ))
consul:
agent:
log_level: (( merge ))
servers:
lan: (( merge ))
ca_cert:
agent_cert:
agent_key:
encrypt_keys:
require_ssl:
server_cert:
server_key:
etcd:
machines: (( merge ))
loggregator_endpoint:
shared_secret: (( merge ))
nats:
user: (( merge ))
password: (( merge ))
machines: (( merge ))
system_domain: (( merge ))
uaa:
clients:
ssh-proxy:
secret: (( merge || nil ))
url: (( merge || nil ))
cf-deployment.yml
#######################################################################
####### VARIABLES
#######################################################################
<%
director_uuid = 'XXX'
static_ip = 'XXX'
root_domain = "#{static_ip}.XXX"
cf_deployment_name = 'XXX'
cf_release_name = 'XXX'
cf_release_version = '0.0.1'
protocol = 'http'
flavorSmall = 'c1.medium'
flavorMedium = 'c1.large'
flavorLarge = 'c1.xlarge'
flavorXlarge = 'c1.xxlarge'
networkInternal = 'XXX'
networkInternalId = 'XXX'
networkInternalCIDR = 'XXX'
networkInternalGateway = 'XXX'
networkInternalSubnetId = 'XXX'
networkInternalDNS = ''
networkExternal = 'XXX'
networkExternalId = 'XXX'
networkExternalSubnetId = 'XXX'
networkExternalCIDR = 'XXX'
networkExternalGateway = 'XXX'
networkExternalDNS = 'XXX, XXX'
networkPublic = 'public'
networkPublicId = 'XXX'
IpInternalPostgres = 'XXX'
IpInternalRouter = 'XXX'
IpInternalNats = 'XXX'
IpInternalEtcd = 'XXX'
IpInternalNfs = 'XXX'
IpInternalConsul1 = 'XXX'
IpInternalConsul2 = 'XXX'
IpInternalConsul3 = 'XXX'
boshStemcell = 'bosh-openstack-kvm-ubuntu-trusty-go_agent-raw'
jobs_doppler_shared_secret = 'XXX'
jobs_uaa_admin_client_secret = 'XXX'
jobs_uaa_batch_password = 'XXX'
jobs_uaa_cc_client_secret = 'XXX'
jobs_uaa_scim_cc_client_secret = 'XXX'
properties_cc_db_encryption_key = 'XXX'
properties_cc_bulk_api_password = 'XXX'
properties_cc_internal_api_password = 'XXX'
properties_cc_staging_upload_password = 'XXX'
uaa_clients_cc_service_dashboards_password = 'XXX'
uaa_clients_cloud_controller_username_lookup_password = 'XXX'
uaa_clients_cc_routing_password = 'XXX'
uaa_clients_gorouter_password = 'XXX'
uaa_clients_login_password = 'XXX'
uaa_clients_servicesmgmt_password = 'XXX'
nats_password = 'XXX'
ccdb_roles_ccadmin_password = 'XXX'
ccdb_roles_uaaadmin_password = 'XXX'
loggregator_endpoint_shared_secret = 'XXX'
metron_endpoint_shared_secret = 'XXX'
%>
---
#######################################################################
####### NETWORK AND MISCELLANEOUS
#######################################################################
name: <%= cf_deployment_name %>
director_uuid: <%= director_uuid %>
releases:
- {name: <%= cf_release_name %>, version: <%= cf_release_version %>}
networks:
- name: <%= networkInternal %>
type: manual
subnets:
- range: <%= networkInternalCIDR %>
gateway: <%= networkInternalGateway %>
dns: [<%= networkInternalDNS %>]
reserved: ["XXX - XXX"]
static: ["XXX - XXX"]
cloud_properties:
net_id: <%= networkInternalId %>
security_groups:
- default
- XXX
- XXX
subnet: networkInternalSubnetId
- name: <%= networkExternal %>
type: manual
subnets:
- range: <%= networkExternalCIDR %>
gateway: <%= networkExternalGateway %>
dns: [<%= networkExternalDNS %>]
reserved: ["XXX - XXX"]
static: ["XXX - XXX"]
cloud_properties:
net_id: <%= networkExternalId %>
security_groups:
- default
- XXX
- XXX
- XXX
subnet: <%= networkExternalSubnetId %>
- name: <%= networkPublic %>
type: vip
cloud_properties:
subnet: []
resource_pools:
- name: common_z1
network: <%= networkInternal %>
stemcell:
name: <%= boshStemcell %>
version: latest
cloud_properties:
instance_type: <%= flavorSmall %>
- name: large_z1
network: <%= networkInternal %>
stemcell:
name: <%= boshStemcell %>
version: latest
cloud_properties:
instance_type: <%= flavorMedium %>
compilation:
workers: 2
network: <%= networkInternal %>
reuse_compilation_vms: true
cloud_properties:
instance_type: <%= flavorMedium %>
update:
canaries: 1
max_in_flight: 32
serial: false
canary_watch_time: 30000-600000
update_watch_time: 30000-600000
#######################################################################
####### JOBS
#######################################################################
jobs:
- name: nats_z1
instances: 1
resource_pool: common_z1
templates:
- {name: nats, release: <%= cf_release_name %>}
- {name: nats_stream_forwarder, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
static_ips: [<%= IpInternalNats %>]
- name: etcd_z1
instances: 1
resource_pool: common_z1
persistent_disk: 102400
templates:
- {name: etcd, release: <%= cf_release_name %>}
- {name: etcd_metrics_server, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
static_ips: [<%= IpInternalEtcd %>]
properties:
etcd_metrics_server:
nats:
machines: [<%= IpInternalNats %>]
password: <%= nats_password %>
username: nats
- name: nfs_z1
instances: 1
persistent_disk: 102400
resource_pool: common_z1
templates:
- {name: debian_nfs_server, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
static_ips: [<%= IpInternalNfs %>]
- name: postgres_z1
instances: 1
persistent_disk: 1024
resource_pool: common_z1
templates:
- {name: postgres, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
static_ips: [<%= IpInternalPostgres %>]
update:
serial: true
- name: api_z1
instances: 1
resource_pool: common_z1
templates:
- {name: cloud_controller_ng, release: <%= cf_release_name %>}
- {name: cloud_controller_worker, release: <%= cf_release_name %>}
- {name: cloud_controller_clock, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
- {name: nfs_mounter, release: <%= cf_release_name %>}
- {name: route_registrar, release: <%= cf_release_name %>}
- {name: consul_agent, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
properties:
nfs_server:
address: <%= IpInternalNfs %>
allow_from_entries: [<%= networkInternalCIDR %>]
route_registrar:
routes:
- name: api
port: 9022
uris:
- "api.<%= root_domain %>"
- name: ha_proxy_z1
instances: 1
resource_pool: common_z1
templates:
- {name: haproxy, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
networks:
- name: <%= networkPublic %>
static_ips: [<%= static_ip %>]
- name: <%= networkExternal %>
default: [gateway, dns]
properties:
ha_proxy:
ssl_pem: |
-----BEGIN CERTIFICATE-----
XXX
-----END CERTIFICATE-----
-----BEGIN RSA PRIVATE KEY-----
XXX
-----END RSA PRIVATE KEY-----
router:
servers:
z1: [<%= IpInternalRouter %>]
- name: hm9000_z1
instances: 1
resource_pool: common_z1
templates:
- {name: hm9000, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
- {name: route_registrar, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
properties:
route_registrar:
routes:
- name: hm9000
port: 5155
uris:
- "hm9000.<%= root_domain %>"
- name: doppler_z1
instances: 1
resource_pool: common_z1
templates:
- {name: doppler, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
properties:
doppler: {zone: z1}
doppler_endpoint:
shared_secret: <%= jobs_doppler_shared_secret %>
- name: loggregator_trafficcontroller_z1
instances: 1
resource_pool: common_z1
templates:
- {name: loggregator_trafficcontroller, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
- {name: route_registrar, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
properties:
traffic_controller: {zone: z1}
route_registrar:
routes:
- name: doppler
port: 8081
uris:
- "doppler.<%= root_domain %>"
- name: loggregator
port: 8080
uris:
- "loggregator.<%= root_domain %>"
- name: uaa_z1
instances: 1
resource_pool: common_z1
templates:
- {name: uaa, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
- {name: route_registrar, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
properties:
login:
catalina_opts: -Xmx768m #-XX:MaxPermSize=256m
route_registrar:
routes:
- name: uaa
port: 8080
uris:
- "uaa.<%= root_domain %>"
- "*.uaa.<%= root_domain %>"
- "login.<%= root_domain %>"
- "*.login.<%= root_domain %>"
uaa:
admin:
client_secret: <%= jobs_uaa_admin_client_secret %>
batch:
password: <%= jobs_uaa_batch_password %>
username: batch_user
cc:
client_secret: <%= jobs_uaa_cc_client_secret %>
scim:
userids_enabled: true
users:
- admin|<%= jobs_uaa_scim_cc_client_secret %>|scim.write,scim.read,openid,cloud_controller.admin,doppler.firehose
uaadb:
address: <%= IpInternalPostgres %>
databases:
- {name: uaadb, tag: uaa}
db_scheme: postgresql
port: 5524
roles:
- {name: uaaadmin, password: <%= ccdb_roles_uaaadmin_password %>, tag: admin}
- name: router_z1
instances: 1
resource_pool: common_z1
templates:
- {name: gorouter, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
static_ips: [<%= IpInternalRouter %>]
properties:
dropsonde: {enabled: true}
- name: runner_z1
instances: 1
resource_pool: common_z1
templates:
- {name: dea_next, release: <%= cf_release_name %>}
- {name: dea_logging_agent, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
properties:
dea_next: {zone: z1}
- name: stats_z1
instances: 1
resource_pool: common_z1
templates:
- {name: collector, release: <%= cf_release_name %>}
- {name: metron_agent, release: <%= cf_release_name %>}
networks:
- name: <%= networkInternal %>
properties:
collector: {deployment_name: <%= cf_deployment_name %>}
- name: consul_z1
instances: 1
resource_pool: common_z1
networks:
- name: <%= networkInternal %>
static_ips: [<%= IpInternalConsul1 %>]
templates:
- {name: consul_agent, release: <%= cf_release_name %>}
# - name: consul_z2
# instances: 0
# networks:
# - name: <%= networkExternal %>
# static_ips: []
#######################################################################
####### Properties
#######################################################################
properties:
networks: {apps: <%= networkInternal %>}
app_domains: [<%= root_domain %>]
cc:
allow_app_ssh_access: false
bulk_api_password: <%= properties_cc_bulk_api_password %>
db_encryption_key: <%= properties_cc_db_encryption_key %>
default_running_security_groups: [public_networks, dns]
default_staging_security_groups: [public_networks, dns]
install_buildpacks:
- {name: java_buildpack, package: buildpack_java}
- {name: ruby_buildpack, package: buildpack_ruby}
- {name: nodejs_buildpack, package: buildpack_nodejs}
- {name: go_buildpack, package: buildpack_go}
- {name: python_buildpack, package: buildpack_python}
- {name: php_buildpack, package: buildpack_php}
- {name: staticfile_buildpack, package: buildpack_staticfile}
- {name: binary_buildpack, package: buildpack_binary}
internal_api_password: <%= properties_cc_internal_api_password %>
quota_definitions:
default:
memory_limit: 4000 #MB
non_basic_services_allowed: false
total_routes: 20
total_services: -1
iron_quota:
memory_limit: 8000 #MB
non_basic_services_allowed: false
total_routes: 100
total_services: 40
bronze_quota:
memory_limit: 16000 #MB
non_basic_services_allowed: false
total_routes: 300
total_services: 300
silver_quota:
memory_limit: 32000 #MB
non_basic_services_allowed: false
total_routes: 1000
total_services: 1000
gold_quota:
memory_limit: 64000 #MB
non_basic_services_allowed: true
total_routes: 5000
total_services: 2000
trial_db_allowed: true
security_group_definitions:
- name: public_networks
rules:
- {destination: XXX-XXX, protocol: all}
- {destination: XXX-XXX, protocol: all}
- {destination: XXX-XXX, protocol: all}
- {destination: XXX-XXX, protocol: all}
- {destination: XXX-XXX, protocol: all}
- name: dns
rules:
- {destination: XXX, ports: '53', protocol: tcp}
- {destination: XXX, ports: '53', protocol: udp}
srv_api_uri: http://api.<%= root_domain %>
staging_upload_password: <%= properties_cc_staging_upload_password %>
staging_upload_user: staging_upload_user
ccdb:
address: <%= IpInternalPostgres %>
databases:
- {name: ccdb, tag: cc}
db_scheme: postgres
port: 5524
roles:
- {name: ccadmin, password: <%= ccdb_roles_ccadmin_password %>, tag: admin}
databases:
databases:
- {name: ccdb, tag: cc, citext: true}
- {name: uaadb, tag: uaa, citext: true}
port: 5524
roles:
- {name: ccadmin, password: <%= ccdb_roles_ccadmin_password %>, tag: admin}
- {name: uaaadmin, password: <%= ccdb_roles_uaaadmin_password %>, tag: admin}
dea_next:
advertise_interval_in_seconds: 5
heartbeat_interval_in_seconds: 10
memory_mb: 33996
description: XXX
domain: <%= root_domain %>
etcd:
machines: [<%= IpInternalEtcd %>]
hm9000:
url: http://hm9000.<%= root_domain %>
logger_endpoint:
port: 4443
loggregator_endpoint:
shared_secret: <%= loggregator_endpoint_shared_secret %>
login:
protocol: http
metron_agent:
zone: z1
deployment: minimal-aws
metron_endpoint:
shared_secret: <%= metron_endpoint_shared_secret %>
nats:
machines: [<%= IpInternalNats %>]
password: <%= nats_password %>
port: 4222
user: nats
nfs_server:
address: <%= IpInternalNfs %>
allow_from_entries: [<%= networkInternalCIDR %>]
ssl:
skip_cert_verify: true
system_domain: <%= root_domain %>
system_domain_organization: default_organization
uaa:
clients:
cc-service-dashboards:
authorities: clients.read,clients.write,clients.admin
authorized-grant-types: client_credentials
scope: openid,cloud_controller_service_permissions.read
secret: <%= uaa_clients_cc_service_dashboards_password %>
cloud_controller_username_lookup:
authorities: scim.userids
authorized-grant-types: client_credentials
secret: <%= uaa_clients_cloud_controller_username_lookup_password %>
cc_routing:
authorities: routing.router_groups.read
secret: <%= uaa_clients_cc_routing_password %>
authorized-grant-types: client_credentials
gorouter:
authorities: clients.read,clients.write,clients.admin,routing.routes.write,routing.routes.read
authorized-grant-types: client_credentials,refresh_token
scope: openid,cloud_controller_service_permissions.read
secret: <%= uaa_clients_gorouter_password %>
doppler:
authorities: uaa.resource
secret: <%= jobs_doppler_shared_secret %>
login:
authorities: oauth.login,scim.write,clients.read,notifications.write,critical_notifications.write,emails.write,scim.userids,password.write
authorized-grant-types: authorization_code,client_credentials,refresh_token
redirect-uri: http://login.<%= root_domain %>
scope: openid,oauth.approvals
secret: <%= uaa_clients_login_password %>
servicesmgmt:
authorities: uaa.resource,oauth.service,clients.read,clients.write,clients.secret
authorized-grant-types: authorization_code,client_credentials,password,implicit
autoapprove: true
redirect-uri: http://servicesmgmt.<%= root_domain %>/auth/cloudfoundry/callback
scope: openid,cloud_controller.read,cloud_controller.write
secret: <%= uaa_clients_servicesmgmt_password %>
jwt:
signing_key: |
-----BEGIN RSA PRIVATE KEY-----
XXX
-----END RSA PRIVATE KEY-----
verification_key: |
-----BEGIN PUBLIC KEY-----
XXX
-----END PUBLIC KEY-----
no_ssl: true
url: http://uaa.<%= root_domain %>
consul:
encrypt_keys:
- XXX
require_ssl: true
ca_cert: |
-----BEGIN CERTIFICATE-----
XXX
-----END CERTIFICATE-----
server_cert: |
-----BEGIN CERTIFICATE-----
XXX
-----END CERTIFICATE-----
server_key: |
-----BEGIN RSA PRIVATE KEY-----
XXX
-----END RSA PRIVATE KEY-----
agent_cert: |
-----BEGIN CERTIFICATE-----
XXX
-----END CERTIFICATE-----
agent_key: |
-----BEGIN RSA PRIVATE KEY-----
XXX
-----END RSA PRIVATE KEY-----
YAML 与顶部的所有变量混淆，然后是 ---。它将其视为文档分隔符，因此基本上只读取 --- 上方的内容。您可以阅读有关 YAML 文档分隔符的更多信息 here。
如果删除第一个三重破折号以上的所有内容，您的 spiff 合并会更进一步，但是由于缺少 consul 属性，您仍然会失败。所以你至少需要添加这些。
即便如此，您也会遇到问题，因为您打算将此工具与完全解析的 CF 部署清单一起用作第三个参数，而不是仍然包含 ERB 的东西（您的 input/cf-deployment.yml 不会转移到 output/config-from-cf.yml。如果您已经对仍然有 ERB 的 CF 清单完成 bosh deploy，您可以通过执行 bosh download manifest XXX input/resolved-cf-deployment.yml 然后 运行:
spiff merge input/config-from-cf.yml \
input/config-from-cf-internal.yml \
input/resolved-cf-deployment.yml \
> output/config-from-cf.yml