Zheng Chu's Blog

让希望永驻


  • 主页

  • 所有专栏

  • 历史文章

  • 标签

  • 关于我

Tensorflow2-ParameterServer

Posted on 2020-10-01 Edited on 2020-12-06 In Tensorflow

数据并行化

整个计算图被保存在一个或多个参数服务器(ps)中。训练操作在多个机器上被执行,这些机器被称作worker。这些worker读取不同的数据(data batches),计算梯度,并将更新操作发送给参数服务器。

数据并行化有两种主要的方案:

  • 同步训练:所有的worker服务器同时读取参数,执行训练操作,等待所有的worker服务器都完成当前训练操作后,梯度被平均后变成一个单独的更新请求并被发送到参数服务器中。所以在任何时候,每个worker服务器看到的计算图参数都是相同的。
Read more »

Tensorflow2-QA

Posted on 2020-10-01 Edited on 2020-12-06 In Tensorflow

What is the difference between keras.evaluate() and keras.predict()?

model.predict的结果是模型的输出y_pred,而model.evaluate返回根据y_pred设置的metrics。

The model.evaluate function predicts the output for the given input and then computes the metrics function specified in themodel.compile and based on y_true and y_pred and returns the computed metric value as the output.

Read more »

Tensorflow2-Dataset

Posted on 2020-10-01 Edited on 2021-03-08 In Tensorflow

tf.data

建立tf.data.Dataset代表数据:建立方法:tf.data.Dataset.from_tensors()或tf.data.Dataset.from_tensor_slices(),或tf.data.TFRecordDataset().

只要建立了Dataset后,就可以像Scala一样进行数据变换生成新的数据;此外,Dataset是可迭代的:

Read more »

Tensorflow2-tf.function

Posted on 2020-10-01 Edited on 2020-12-06 In Tensorflow

@tf.function弥补Eager execution带来的效率问题:

  • Debug in eager mode, then decorate with @tf.function.
  • Don’t rely on Python side effects like object mutation or list appends.
  • tf.function works best with TensorFlow ops; NumPy and Python calls are converted to constants.
Read more »

Tensorflow2-highLevel

Posted on 2020-10-01 Edited on 2020-12-06 In Tensorflow
  • GAN training loop from scratch
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
discriminator = keras.Sequential(
[
keras.Input(shape=(28, 28, 1)),
layers.Conv2D(64, (3, 3), strides=(2, 2), padding="same"),
layers.LeakyReLU(alpha=0.2),
layers.Conv2D(128, (3, 3), strides=(2, 2), padding="same"),
layers.LeakyReLU(alpha=0.2),
layers.GlobalMaxPooling2D(),
layers.Dense(1),
],
name="discriminator",
)
discriminator.summary()


latent_dim = 128

generator = keras.Sequential(
[
keras.Input(shape=(latent_dim,)),
# We want to generate 128 coefficients to reshape into a 7x7x128 map
layers.Dense(7 * 7 * 128),
layers.LeakyReLU(alpha=0.2),
layers.Reshape((7, 7, 128)),
layers.Conv2DTranspose(128, (4, 4), strides=(2, 2), padding="same"),
layers.LeakyReLU(alpha=0.2),
layers.Conv2DTranspose(128, (4, 4), strides=(2, 2), padding="same"),
layers.LeakyReLU(alpha=0.2),
layers.Conv2D(1, (7, 7), padding="same", activation="sigmoid"),
],
name="generator",
)


# Instantiate one optimizer for the discriminator and another for the generator.
d_optimizer = keras.optimizers.Adam(learning_rate=0.0003)
g_optimizer = keras.optimizers.Adam(learning_rate=0.0004)

# Instantiate a loss function.
loss_fn = keras.losses.BinaryCrossentropy(from_logits=True)


@tf.function
def train_step(real_images):
# Sample random points in the latent space
random_latent_vectors = tf.random.normal(shape=(batch_size, latent_dim))
# Decode them to fake images
generated_images = generator(random_latent_vectors)
# Combine them with real images
combined_images = tf.concat([generated_images, real_images], axis=0)

# Assemble labels discriminating real from fake images
labels = tf.concat(
[tf.ones((batch_size, 1)), tf.zeros((real_images.shape[0], 1))], axis=0
)
# Add random noise to the labels - important trick!
labels += 0.05 * tf.random.uniform(labels.shape)

# Train the discriminator
with tf.GradientTape() as tape:
predictions = discriminator(combined_images)
d_loss = loss_fn(labels, predictions)
grads = tape.gradient(d_loss, discriminator.trainable_weights)
d_optimizer.apply_gradients(zip(grads, discriminator.trainable_weights))

# Sample random points in the latent space
random_latent_vectors = tf.random.normal(shape=(batch_size, latent_dim))
# Assemble labels that say "all real images"
misleading_labels = tf.zeros((batch_size, 1))

# Train the generator (note that we should *not* update the weights
# of the discriminator)!
with tf.GradientTape() as tape:
predictions = discriminator(generator(random_latent_vectors))
g_loss = loss_fn(misleading_labels, predictions)
grads = tape.gradient(g_loss, generator.trainable_weights)
g_optimizer.apply_gradients(zip(grads, generator.trainable_weights))
return d_loss, g_loss, generated_images



import os

# Prepare the dataset. We use both the training & test MNIST digits.
batch_size = 64
(x_train, _), (x_test, _) = keras.datasets.mnist.load_data()
all_digits = np.concatenate([x_train, x_test])
all_digits = all_digits.astype("float32") / 255.0
all_digits = np.reshape(all_digits, (-1, 28, 28, 1))
dataset = tf.data.Dataset.from_tensor_slices(all_digits)
dataset = dataset.shuffle(buffer_size=1024).batch(batch_size)

epochs = 1 # In practice you need at least 20 epochs to generate nice digits.
save_dir = "./"

for epoch in range(epochs):
print("\nStart epoch", epoch)

for step, real_images in enumerate(dataset):
# Train the discriminator & generator on one batch of real images.
d_loss, g_loss, generated_images = train_step(real_images)

# Logging.
if step % 200 == 0:
# Print metrics
print("discriminator loss at step %d: %.2f" % (step, d_loss))
print("adversarial loss at step %d: %.2f" % (step, g_loss))

# Save one generated image
img = tf.keras.preprocessing.image.array_to_img(
generated_images[0] * 255.0, scale=False
)
img.save(os.path.join(save_dir, "generated_img" + str(step) + ".png"))

# To limit execution time we stop after 10 steps.
# Remove the lines below to actually train the model!
if step > 10:
break
Read more »

Tensorflow2-Estimator

Posted on 2020-10-01 Edited on 2020-12-06 In Tensorflow

Estimator封装了四个主要功能:

  • training
  • evaluation
  • prediction
  • export for serving

Esitmator提供了现在tf.keras正在构建中的功能:

  • Parameter server based training
  • Full TFX integration.
Read more »

电影🎬记录.md

Posted on 2020-09-13 Edited on 2020-12-06 In 世界

今天创建了名为《世界》的文件夹,把我想记录的,包括看过的电影,书籍,旅游等等,都放在这里。

《教父》

1、麦克·柯里昂:

  • 个性:
    • 谨慎冷静:麦克探望病床上的父亲,发现周围的手下都被警察支走,立马打电话叫人过来,并找护士商量,给父亲换了病房,并且安排前来探望教父的糕点师,在门口伪装成携qiang的手下上演空城计,成功欺骗了刺杀的敌人,为自己人过来争取了时间。(可以发现,糕点师事后颤抖的手几乎握不住烟,而麦克非常冷静。)
    • 原则与手段:肃清了家族外的所有外敌,包括背叛自己的亲人,通过手段给背叛者留一丝希望,再引诱背叛者坦白自己的背叛,得到有价值的信息后,立马绞死了叛徒。
    • 胆识:独自一人刺杀了警官与做卖毒生意的首领;
    • 敢爱:自己喜欢的,立马行动;第一二任妻子都如此。
    • 不动声色:教父曾告诫麦克:“谁主动提出和巴西尼谈判,谁就是叛徒”。教父的葬礼上,泰西欧这个老将走近麦克,提出了谈判。麦克马上心知肚明,但表面上,仍不动声色,一口答应了谈判。
Read more »

Hadoop学习记录

Posted on 2020-07-13 Edited on 2020-12-06 In 大数据

前言

从 2003 年到 2006 年,Google 分别在 ODSI 与 SOSP 发表了 3 篇论文,引起了业界对于分布式系统的广泛讨论,这三篇论文分别是:

SOSP2003:The Google File System;

ODSI2004:MapReduce: Simplifed Data Processing on Large Clusters;

Read more »

spark基础1

Posted on 2020-07-13 Edited on 2020-12-06 In 大数据

一、大数据分析

Spark 继承了 MapReduce 的线性扩展性和容错性,同时对它做了一些重量级扩展。

差异:Spark 摒弃了 MapReduce 先 map 再 reduce这样的严格方式,Spark 引擎可以执行更通用的有向无环图(directed acyclic graph,DAG)算子。这就意味着,在 MapReduce 中需要将中间结果写入分布式文件系统时,Spark 能将中间结果直接传到流水作业线的下一步。

Read more »

Docker记录

Posted on 2020-07-12 Edited on 2020-12-06 In 工具

概念

Docker架构:客户端——服务器模式

客户端 + 服务器(守护进程) + 注册处(可选)

注册处:存储Docker映像和映像的元数据。

服务器(守护进程):可以在任意多个服务器中运行,作用是构建、运行、管理容器。

客户端:告诉服务器做什么

Docker 包括三个基本概念:

  • 镜像(Image):一种底层定义,指明把什么放入容器中,是容器的文件系统。

    可以使用远程仓库中别人制作好的镜像文件,也可以自己制作镜像文件。要制作镜像文件就要编写 Dockerfile 文件,其类似于 Makefile 文件。

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    # 列出本机的所有image文件
    docker image ls
    docker images

    # 删除指定的image文件
    docker image rm <image-name>
    docker rmi <image-name>

    # 将指定的image文件从远程仓库拉取到本地
    docker image pull <image-name>[:tag]
    docker pull <image-name>[:tag]

    # 利用当前文件夹中的Dockerfile制作一个名为demo、tag为0.0.1的image文件
    # 若不指定tag,则默认的标签为latest
    docker image build -t demo:0.0.1 .
    docker build -t demo:0.0.1 .
Read more »
<123…9>
Zheng Chu

Zheng Chu

90 posts
20 categories
25 tags
GitHub 简书 CSDN E-Mail
© 2021 Zheng Chu
Powered by Hexo v4.2.1
|
Theme – NexT.Pisces v7.3.0
|