Official Google Cloud Certified Associate Cloud Engineer
Auteur: Dan Sullivan
Ma note: 7/10
Mes highlights
The fl ashcards in the TestBank will push the limits of what you should know for the cer-tif i cation exam
log sinks to export logs to external systems
Autoscalers can add or remove VMs from the cluster based on the workload. This is called autoscaling.
Managed clusters make use of containers. A container is like a lightweight VM that isolates processes running in one container from processes running in another container on the same server.
Google Cloud Platform has two serverless computing options: App Engine and Cloud Functions.
Objects are grouped into buckets. Each object is individually addressable, usually by a URL.
With a block storage system, you can install fi le systems on top of the block storage, or you can run applications that access blocks directly. Some relational databases can be designed to access blocks directly rather working through fi le systems.
In Linux fi le systems, 4KB is a common block size. Relational databases often write directly to blocks, but they often use larger sizes, such as 8KB or more.
Your internal GCP network is def i ned as a virtual private cloud (VPC).
File systems and databases make use of block storage systems
IaaS computing product is called Compute Engine, and the PaaS offerings are App Engine and Cloud Functions
KVM stands for Kernel Virtual Machine and provides virtualization on Linux systems running on x86 hardware
It will frequently be shut down if the preemptible VM has run for at least 24 hours.
Kubernetes Engine is a GCP product that allows users to describe the compute, storage, and memory resources they’d like to run their services
App Engine is well suited for web and mobile backend applications.
App Engine is available in two types: standard and fl exible.
In the fl exible environment, you run Docker containers in the App Engine environment. The fl exible environment works well in cases where you have application code but also need libraries or other third-party software installed
Cloud Storage is GCP’s object storage system
Cloud Storage is useful for storing objects that are treated as single units of data. For example, an image fi le is a good candidate for object storage
nearline storage is a good option because it costs less than regional or multiregional storage and is optimized for infrequent access.
you could def i ne a policy that moves all objects more than 60 days old in a bucket to nearline storage or deletes any object in a coldline storage bucket that is older than fi ve years.
The Cloud Storage for Firebase API is designed to provide secure transmission as well as robust recovery mechanisms to handle potentially problematic network quality
Filestore implements the Network File System (NFS) protocol so system administrators can easily mount shared fi le systems on virtual servers
Storage systems like the ones just described are used to store coarse-grained objects, like fi les. When data is more fi nely structured and has to be retrieved using query languages t
Cloud Datastore is a NoSQL document databas
Product catalogs, user prof i les, and user navigation history are examples of the kinds of applications that use Cloud Datastore.
Cloud Memorystore is a managed Redis service for caching frequently used data in memory. Caches like this are used to reduce the time needed to read data into an application. Cloud Memorystore is designed to provide submillisecond access to data.
Although multiple enterprises will use the same cloud infrastructure, each enterprise can logically isolate its cloud resources by creating a virtual private cloud (VPC).
Cloud Armor features include the following: ■ Ability to allow or restrict access based on IP address ■ Predefined rules to counter cross-site scripting attacks ■ Ability to counter SQL injection attacks ■ Ability to define rules at both level 3 (network) and level 7 (application) ■ Allows and restricts access based on the geolocation of incoming traffic
CDNs are especially important for sites with large amounts of static content and a global audience
it could use Partner Interconnect. This service depends on a third-party network provider to provide connectivity between the company’s data center and a Google facility.
Google offers VPN services that enable traff i c to transmit between data centers and Google facilities using the public Internet.
Management tools are designed for DevOps professionals who are responsible for ensuring the reliability, availability, and scalability of applications.
The Apigee platform allows developers to deploy, monitor, and secure their APIs. It also generates API proxies based on the Open API Specif i cation.
BigQuery, a petabyte-scale analytics database service for data warehousing ■ Cloud Dataflow, a framework for defining batch and stream processing pipelines ■ Cloud Dataproc, a managed Hadoop and Spark service ■ Cloud Dataprep, a service that allows analysts to explore and prepare data for analysis
Relational databases have been traditionally diff i cult to horizontally scale
Spanner is a global relational database that provides the advantages of relational databases with the scalability previously found only in NoSQL databases.
NoSQL databases are designed to be horizontally scalable
NoSQL databases may be key-value stores like Cloud Memorystore, document databases like Cloud Datastore, or wide-column databases such as Cloud Bigtable.
A single cloud identity is associated with at most one organization
One way to think of the difference is that IAM specif i es who can do things, and the Organization Policy Service specif i es what can be done with resources.
Constraints are restrictions on services
For example, if you want to deny access to serial ports on VMs, you can set constraints/compute.disableSerialPortAccess to TRUE
There are three types of roles in Google Cloud Platform: ■ Primitive roles ■ Predefined roles ■ Custom roles
It is important to know that permissions cannot be assigned to users. They can be assigned only to roles. Roles are then assigned to users.
two types of service accounts, user-managed service accounts and Google- managed service accounts
Service accounts are created automatically when resources are created
Custom images are especially useful if you have to conf i gure an operating system and install additional software on each instance of a VM that you run
Compute Engine Security Admin Users with this role can create, modify, and delete SSL certif i cates and fi rewall rules.
Preemptible VMs are short-lived compute instances suitable for running certain types of workloads— particularly for applications that perform fi nancial modeling, rendering, big data, continuous integration, and web crawling operations.
Install software packages or custom libraries. ■ Have fine-grained control over which users have permissions on the instance. ■ Have control over SSL certificates and firewall rules for the instance.
In some ways, the App Engine fl exible environment is similar to the Kubernetes Engine, which will be discussed in the next section
With Kubernetes Engine you have control over your cluster but must monitor and manage that cluster using tools such as Stackdriver monitoring and autoscaling. With the App Engine fl exible environment, the health of App Engine servers is monitored by Google and corrected as needed without any inter-vention on your part.
If you wanted, you could deploy a set of VMs, install Kubernetes on your VMs, and manage the Kubernetes platform yourself. With Kubernetes Engine you get the benef i ts of Kubernetes without the adminis-trative overhead.
When a resource is consumed beyond the threshold, then Kubernetes will start shutting down pods
Kubernetes Engine is a good choice for large-scale applications that require high availabil-ity and high reliability
We should not have to keep track of dependencies between services if we can avoid it. Cloud Functions helps us avoid that situation.
Generally, with more control comes more responsibility and management overhead
Let’s create a VM in Compute Engine. We have three options for doing this: we can use Google Cloud Console, Google Cloud SDK, or Google Cloud Shell.
Labels are particularly important when your number of servers grows. It is a best practice to include a description and labels for all VMs.
project-wide SSH keys, which are used to give users project-wide access to VMs. You can block that behavior at the VM if you use project-wide SSH keys and do not want all project users to have access to this machine.
If you need to ensure that your VMs run on a server only with your other VMs, then you can specify sole tenancy
The most common way is to use SSH when logging into a Linux server or Remote Desktop Protocol (RDP) when logging into a Windows server.
Use the console for ad hoc administration of VMs. Use scripts with gcloud commands for tasks that will be repeated.
Use startup scripts to perform software updates and other tasks that should be per-formed on startup.
before you create the snapshot; other-wise, data in memory that should be written to disk may be lost. The way to fl ush the disk buffers will vary by application. For example, MySQL has a FLUSH statement.
It is a good practice to label all resources with a consistent labeling convention
snapshots are used to make data available on a disk, while images are used to create VMs
The –async parameter displays information about the start operation. The –verbose option in many Linux commands provides similar functionality
two types of instance groups: managed and unmanaged.
You can conf i gure an autoscaling policy to trigger adding or removing instances based on CPU utilization, monitoring metric, load-balancing capacity, or queue-based workloads.
Instance groups are sets of instances managed as a single entity. Instance group templates specify the conf i guration of an instance group and the instances in it. Managed instance groups support autoscaling and load balancing.
Nodes are VMs that run containers conf i gured to run an application
The nodes run an agent called kubelet, which is the service that communicates with the cluster master.
This structure is designed to support running one instance of an application within the cluster as a pod.
ReplicaSet will detect that not enough pods for that application or workload are running and will create another.
StatefulSets are like deployments, but they assign unique identif i ers to pods. This enables Kubernetes to track which pod is used by which client and keep them together.
ReplicaSets are controllers for ensuring that the correct number of pods are running.
Pending, which indicates the pod is downloading images
Failed, which indicates at least one container failed; and Unknown, which means the master cannot reach the node and status cannot be determined.
To list information about nodes and pods, use the kubectl command. First, you need to ensure you have a properly conf i gured kubeconfig fi le, which contains information on how to communicate with the cluster API. Run the command gcloud container clusters get-credentials with the name of a zone or region and the name of a cluster
You can list the nodes in a cluster using the following: kubectl get nodes
In this way, you can maintain multiple versions of your application at one time, which is especially helpful for testing new features on a small number of users before rolling the change out to all users
The app is written in Python, so you’ll use the Python runtime in App Engine.
App Engine provides three ways to split traff i c: by IP address, by HTTP cookie, and by random selection
Memorystore, a managed Redis service
When you need to store large volumes of data, that is, up to exabytes, and share it widely, object storage is a good option
It is impor-tant to remember that buckets share a global namespace
Using Cloud Storage Fuse, you can down-load and upload fi les to buckets using fi le system commands, but it does not provide full fi le system functionality.
Versioning is useful when you need to keep a history of changes to an object or want to mitigate the risk of accidentally deleting an object.
Caches, like Memorystore
three broad categories of data models available in GCP: object, relational, and NoSQL
If you need to update an object, you must copy it to a server, make the change, and then copy the updated version back to the object storage system.
This data model is well suited for archived data, machine learning training data, and old Internet of Things (IoT) data that needs to be saved but is no longer actively analyzed.
Relational databases, like Cloud SQL and Cloud Spanner, support database trans-actions. A transaction is a set of operations that is guaranteed to succeed or fail in its entirety
Cloud SQL is a managed database service that provides MySQL and PostgreSQL databases. Cloud SQL is used for databases that do not need to scale horizontally, that is, by adding additional servers to a cluster. Cloud SQL databases scale vertically, that is, by running on servers with more memory and more CPU. Cloud Spanner is used when you have extremely large volumes of relational data or data that needs to be globally distributed while ensuring consistency and transaction integrity across all servers.
Cloud Spanner for applications like global supply chains and fi nancial services applications, while Cloud SQL is often used for web applications, busi-ness intelligence, and ecommerce applications
BigQuery is a service designed for a data warehouse and analytic applications. BigQuery is designed to store petabytes of data. BigQuery works with large numbers of rows and col-umns of data and is not suitable for transaction-oriented applications
GCP has three NoSQL options: ■ Cloud Datastore ■ Cloud Firestore ■ Cloud Bigtable
Not all rows need to use all columns, so in that way it is like Datastore—neither require a fi xed schema to structure the data.
mb” is short for “make bucket.”
The SQL output is useful if you plan to import the data to another relational database. CSV is a good choice if you need to move this data into a nonrelational database.
Avro is a compact binary format that supports complex data structures. When data is saved in the Avro format, a schema is written to the fi le along with data
command-line tool for working with BigQuery is bq
Cloud Bigtable does not have an Export and Import option in the Cloud Console or in gcloud. You have two other options: using a Java application for importing and exporting or using the HBase interface to execute HBase commands.
Cloud Dataproc is not a database like Cloud SQL or Bigtable; rather, it is a data analysis platform
Cloud Dataproc is not designed to be a persistent store of data
Bigtable and Cloud Dataproc, have command-line options only.
Cloud Spanner uses the Dataf l ow service for importing and exporting
You can use the gsutil acl ch command to change permissions on a Cloud Storage bucket.
A Java program is run from the command line to import or export data from Bigtable. Cloud Dataproc is different in that it is not designed as a persistent data store
GCP auto-matically creates a VPC when you create a project
The shared VPC is hosted in a common project.
Priority: Highest-priority rules are applied; any rule with a lower priority that matches are not applied. Priority is specified by an integer from 0 to 65535. 0 is the highest priority, and 65535 is lowest.
If no protocol is specified, then the rule applies to all protocols.
Private zones respond only to queries that originate from resources in the same project as the zone.
DNSSEC is designed to prevent spoof i ng (a client appearing to be some other client) and cache poisoning (a client sending incorrect informa-tion to update the DNS server).
DNS Forwarding is now available, which allows your DNS queries to be passed to an on-premise DNS server if you are using Cloud VPN or Interconnect.
The Internal TCP/UDP load balancer is the only internal load balancer. The HTTP(S), SSL Proxy, TCP Proxy, and Network TCP/UDP load balancers are all external
Marketplace is another name for the Cloud Launcher page in Cloud Console
Google recommends using Python to create template fi les unless the templates are relatively simple, in which case it is appropriate to use Jinja2.
The launch stage options are as follows: Alpha, Beta, General Availability, and Disabled.
Scopes authorize the access to API methods.
To conf i gure access controls for a VM, you will need to conf i gure both IAM roles and scopes.
An instance can only perform operations allowed by both IAM roles assigned to the service account and scopes def i ned on the instance
The three primitive roles are owner, editor, and viewer
alerts based on resource metrics and custom met-rics
Stackdriver works in hybrid environments with support for GCP, Amazon Web Services, and on-premise resources.
A policy consists of conditions that determine when to issue an alert or notif i cation
This process of grouping data into regular-sized buckets of time is called aligning
two ways to create custom metrics: using OpenCensus, an open source monitoring library (https://opencensus.io/) or using Stackdriver’s Monitoring API.
Stackdriver Logging retains log data for 30 days
the location to which you write the log data is called a sink.