Modules

1 Digital Transformation with Google Cloud

• 'why ' we do what we do .
• Cloud types and tips
  • data cloud - manage data across the entire data lifecycle, with AI built in.
  • open infrastructure cloud - innovate and scale from on premises, to edge to cloud.
  • open standard - standard (specifications)
  • open source - everyone can see the source code
  • collaboration cloud - teams, google workspace
  • trusted cloud - with security tools
• CapEx to OpEx - > Capital expenses (own servers ON PREMISES, ) . OpEx (only pay what you use CLOUD)
• data sovereignty - right to be forgotten
• data residency - data must stay in the place where it was created

NETWORKS

Internet Protocols -
#️⃣ 🔤 📒
IP Address =====> Domain Name =====> DNS (Domain Name System ) (phone book)
35.24.123.7====>www.myblog.com ====> www.myblog.com = 35.24.123.7
/\ /\

bandwidth - how much data can be transferred in a certain amount of time. theoretical maximum.
throughput - actual how much data can be transferred in a certain amount of time. real world network limitations.
latency - how long does it take for a request to reach end point
Network's Edge - entry point to the network . More edges than zones

[[Cloud VPNs]] [[Cloud Interconnect]] method to connect networks together

2 Exploring data transformation with Google cloud

types of data

no analysis -> day to day operations.

Structured data - SQL (relational databases) - Tables (rows and columns), well defined schema. [Cloud SQL], [Cloud Spanner]

Semi-structured Data - NoSQL (non-relational databases) - diverse data types, not a tabular format . emails, messages . [Firestore] [Cloud Bigtable] .

Unstructured Data - Images, video, files, mp3 , backups. [[Cloud Storage]] -> standard🔥, nearline(30), coldline(90), archive(365). Autoclass - transitions objects automatically to appropriate storage classes if you haven't looked at them in a while.

types of data storage places

**databases** - SQL [[Cloud SQL]] / [[Cloud Spanner]] , NoSQL [[Firestore]] / [[Cloud Bigtable]]
**data warehouses** - analyze trends (reports), market analysis. [BigQuery]
**data lakes** - have all content in one place. A repository to ingest, store, explore, process and analyze any type or volume of raw data. All storage products.

MIGRATION

• lift and shift - don’t change anything in it . Use managed services [[Datastream]] - upload data continuously for migration doesn’t , [[DMS Database Migration service ]].
• [[Looker]] - to visualize our data. Business intelligence platform to understand our data. 📈 is like [[BigQuery]] you can connect any sql database source to looker

ANALYTICS IN DATA
batch processing - payroll
Streaming data - inventory - needs to stay up to date [Dataflow] , [PubSub]

[[Etl]] extract it , transform it, load it. [[data pipeline]] process.
[[Apache Beam]] - open source programming model for [[ data pipeline]] design. You can use[[Dataflow]] in Google for a serverless fully managed experience and not have to do everything in [[Apache Beam]].

3 innovating with AI

AI

ML is a subset of AI . 🤖

How ai and ml (forward looking , future data) differ from data analytics / business intelligence (historic backward looking data)

Completeness- all data required
Uniqueness - no dupes
Timeliness- is data old or fresh
Validity- formatting
Accuracy ✅❌ - true or not
Consistency 🧑🏼‍✈️ - data is uniform, and not contradictory (marine called differently in different data sources by name in some and social security number in others)

——-
Googles AI principles for safety

• testable and tested
• Privacy
• Accountable to people
• For uses that support the principles
• Bias reinforcing
• Socially beneficial
• high standards of scientific excellence

Google will not design or deploy

• no surveillance
• no crimes
• No weapons
• Causes harm

If you need to give it some personal info - create a storage bucket and tell paid Gemini to only use that info and not feed it into the internet .

CLOUD PRODUCTS FOR AI AND ML

• [[BIGQUERY ML ]] - you have your own data, train your own model. SQL , predict (future) integrates with [[Vertex AI]] , platform to deploy the model registry for an endpoint, custom built model as an app .
• [[Pre-trained APIs gOOGLE AI ]] - > if you DON'T have your own training data or a data scientist.
• [[AutoML]] - Takes your data and trains the pre trained models -> you load your data and it chooses the best [[Machine learning model]] for you . Riding on the shoulders of giants.
• Custom training -> [[Vertex AI]] - suite of products to help each stage of the ml workflow -> Gather data, feature training , building models , deploying and monitoring models. [[TensorFlow]] - training and inference of neural networks, created by Google. for researchers to innovate .

==============
Security operations (SecOps). Practice that is all about protecting your organization's data and systems in the cloud. It involves a combination of processes and technologies that help reduce the risk of data breaches, system outages, and other security incidents.

Site reliability engineering (SRE) - ensures the reliability, availability, and efficiency of software systems and services deployed in the cloud.

Zero trust security - a strategic framework that establishes strict access controls based on the principle of continuous verification. Security operations focuses on the practical, day-to-day implementation of security measures, like threat detection, incident response, and monitoring.

Cloud security posture management (CSPM) - specifically focuses on identifying and correcting misconfigurations or vulnerabilities within your cloud infrastructure to maintain a strong security posture in the cloud.

==============

Cloud Profiler tool- It identifies how much CPU power, memory, and other resources an application uses . It's designed to analyze application code and pinpoint areas where resources (CPU, memory) are inefficiently used, contributing to performance bottlenecks.

Cloud Monitoring - big-picture health of infrastructure and services . comprehensive view of your cloud infrastructure and applications.

4 Modernize infrastructure and applications with google cloud

Microservices - services communicate through APIs . and its more when we are talking about architecture and design . modern cloud app development
Monolith - opposite of microservices, everything is tightly coupled and can't scale independently.
Container - app + dependencies
[[Virtual Machine]] - container + Operating System

[[Kubernetes Engine (Google)]] - Orchestrates containers . - managing infrastructure, complex dependencies between infrastructure.

Serverless - you just provide the code. Google does everything else.

Serverless Computing Products
[[App Engine]] - build and deploy web applications (containerized)
[[Cloud Run]] fully managed environment for running containerized apps that can handle multiple events at the same time. / [[Cloud Functions]] simple, single purpose event-driven functions.

potential drawbacks to rehosting on prem legacy services to the cloud

rehosting legacy apps.

[[Google Cloud VMWare Engine ]] - migrate existing VMWare workloads

[[Bare Metal Solution]] - for ORACLE workloads.

5 Trust and security with google cloud

[[Apigee Edge]] - manage APIs

5 security ===================

Privileged access - grants certain users a broader access than most users
Least privilege - only access needed
Zero trust - assumes nothing and no one can be trusted by default

Security by default - security from the start
Security posture - overall security status of a cloud environment
Cyber resilience - an organization’s ability to withstand and recover quickly from cyber attacks.

Firewall - network device that regulates traffic based on security rules
Encryption - converting data to unreadable by using an encryption algorithm
Decryption - uses an encryption 🔑 to restore encrypted data to original form

CIA - confidentiality , integrity, availability

3 As of cloud identity management - Authentication, authorization, auditing

============
(dAY 2, MINUTES INTO VIDEO)
Network safety
5:01 -> 5:06 =

5:17-> 5:23 =

5:21 -> 5:27 =

Compliance - create organization policy constraints + IAM access to store data in the right region 🗾 (5:43 day 2)

6 Scaling with Google cloud operations

MODULE 6
6:01 - scaling with GOogle cloud - >

4 🌟golden signals measure performance and reliability - ⏰ latency check, 🚗 traffic , saturation, ❌ errors

High availability- remain operational even if software or hardware issues occur.

Key Design Principles

Redundancy - duplicate critical components
Replication - several copies across the different regions.
Regions, scalable infrastructure, backups

Observability Tools

[[Google Cloud Monitoring]] - metrics , NUMBERS . how latent. how many hits, how many users logged in . SRE team .
[[Google Cloud Logging]] - details - somebody hit your endpoint, here is their IP address, this is the request, this is the response .
[[Google Cloud trace]] - app visibility - slow or not - LATENCY
[[Google Cloud profiler]] - app visibility - MEMORY USAGE
[[Google Cloud Error Reporting]] - app visibility - CRASHES , ERRORS AND HOW OFTEN

Levels of Support

Basic - free
Standard Support
Enhanced
Premium support

=========
Notes from Test prep

Dataflow is used to transform and process data after it is received, not to ingest it.

Pub/Sub is a messaging service that can receive data from device streams such as sensors, at the start of a data pipeline.

Cloud Billing reports offer a reactive method to help you track and understand what you’ve already spent on Google Cloud resources and provide ways to help optimize your costs

[[Bare Metal Solution]] - for Oracle workloads

[[Dataproc]]-  is a managed service for large-scale data processing using Apache Hadoop and Spark. While relevant for data preparation for AI, it's not focused on the model development itself.

Methods to connect networks
[[Cloud VPNs]]
[[Cloud Interconnect]]

[[Cloud Run]] - runs containerized web applications