ValueError: Found unexpected losses or metrics that do not correspond to any Model output
ValueError: Found unexpected losses or metrics that do not correspond to any Model output
我正在使用具有 1 个特征列（字符串）和 97 个标签列（多标签分类）的 CSV 数据集，每行 1 或 0。
数据：
| | vectorizer_input | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 |
|---:|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|
| 0 | want improve question update question on-topic stack overflow closed 8 year ago json need update {activity_code1 activity_namefootball} {activity_code1 activity_nametennis} based activity_code achieve php first need decode change data re-encode save back file | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 1 | string xml need return xml document default returned content type text plain content rendered need content type application xml enabled option respectbrowseracceptheader serialize object xml set correct content type except object string string xml need return xml document return contentresult controller startup project.json response github good measure | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 2 | volumetric imaging data consisting value sampled regular grid x z non-cubic voxel shape space adjacent point z greater x would eventually like able interpolate value arbitrary 2d plane pass volume like aware scipy.ndimage.map_coordinates case using le straightforward implicitly assumes spacing element input array equal across dimension could first resample input array according smallest voxel dimension voxels would cube use map_coordinates interpolate plane seem like great idea interpolate data twice also aware scipy various interpolators irregularly-spaced nd data linearndinterpolator nearestndinterpolator etc. slow memory-intensive purpose best way interpolating data given know value regularly spaced within dimension use map_coordinates little bit algebra let say spacing grid dx dy dz need map real world coordinate array index coordinate let define three new variable array index input map_coordinates array shape ... number dimension original data define array transform real world coordinate array index coordinate dividing scaling little broadcasting magic put together example let say want interpolate value along plane 2*y - z = 0 take two vector perpendicular plane normal vector get coordinate want interpolate convert array index coordinate interpoalte using map_coordinates last array shape 10 10 position [u_idx v_idx] value corresponding coordinate coords[ u_idx v_idx] could build idea handle interpolation coordinate start zero adding offset scaling | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 3 | best way calculate sum matrix a^i + a^ i+1 + a^i+2........a^n large n thought two possible way 1 use logarithmic matrix exponentiation lme a^i calculate subsequent matrix multiplying problem really take advantage lme algorithm using lowest power 2 use lme finding a^n memoize intermediate calculation problem much space required large n third way notice let logarithmic number step need compute inverse direct implementation lead log n ^2 factor keep log n computing power compute b | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 4 | named color available matplotlib use plot find list matplotlib documentation claim name however found color also used least context list anyone know exhaustive list named color available constantly forget name color want use keep coming back question = previous answer great find bit difficult get overview available color posted image prefer color grouped similar color slightly tweaked matplotlib answer mentioned comment get color list sorted column order identical would sort eye think give good overview updated image code reflect arebeccapurple added three sage color moved xkcd prefix since posted answer originally really change much matplotlib example code completeness updated 2017-10-25 merged previous update section would like use additional named color plotting matplotlib use xkcd crowdsourced color name via xkcd prefix access plethora named color default tableau color available matplotlib via notab prefix ten distinct color also plot color html hex code similar specifying rgb tuple rather named color apart fact hex code passed string include image 16 million color choose from.. detail please refer matplotlib color documentation source file specifying available color _color_data.py | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
进口：
import re
import tensorflow as tf
from nltk.corpus import stopwords
from datasets import load_dataset
from nltk.stem import WordNetLemmatizer
import numpy as np
from tensorflow.keras import models, layers, losses, metrics
from sklearn.metrics import classification_report
我如何加载数据集：
feature_cols = ['vectorizer_input']
label_cols = [str(i) for i in range(97)]
tfdataset = tf.data.experimental.make_csv_dataset(
name,
batch_size=128,
column_names=feature_cols + label_cols,
column_defaults=[tf.dtypes.string] + [tf.dtypes.int64] * 97,
shuffle=False
)
然后我指定多个标签列：
def make_features_labels_tuple(x, feature_cols, label_cols):
labels = {k: x[k] for k in x if k in label_cols}
features = {k: x[k] for k in x if k in feature_cols}
return features, labels
tfdataset = tfdataset.map(lambda x: make_features_labels_tuple(x, feature_cols, label_cols))
这是我现在的简单模型：
vectorizer = layers.TextVectorization(max_tokens=4000, output_sequence_length=4000)
model = models.Sequential(
[
vectorizer,
layers.Embedding(4000, 64),
layers.GlobalMaxPool1D(),
layers.Dense(97, activation='sigmoid')
]
)
model.compile(
optimizer='adam',
loss=losses.BinaryCrossentropy(),
metrics=[metrics.Accuracy(), metrics.Precision(), metrics.Recall()]
)
模型摘要：
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
text_vectorization_44 (Text (None, 4000) 0
Vectorization)
embedding_16 (Embedding) (None, 4000, 64) 256000
global_max_pooling1d_16 (Gl (None, 64) 0
obalMaxPooling1D)
dense_16 (Dense) (None, 97) 6305
=================================================================
Total params: 262,305
Trainable params: 262,305
Non-trainable params: 0
_________________________________________________________________
这是我在尝试 model.fit:
时遇到的错误
ValueError: Found unexpected losses or metrics that do not correspond to any Model output: dict_keys(['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '20', '21', '22', '23', '24', '25', '26', '27', '28', '29', '30', '31', '32', '33', '34', '35', '36', '37', '38', '39', '40', '41', '42', '43', '44', '45', '46', '47', '48', '49', '50', '51', '52', '53', '54', '55', '56', '57', '58', '59', '60', '61', '62', '63', '64', '65', '66', '67', '68', '69', '70', '71', '72', '73', '74', '75', '76', '77', '78', '79', '80', '81', '82', '83', '84', '85', '86', '87', '88', '89', '90', '91', '92', '93', '94', '95', '96']). Valid mode output names: ['dense_7']. Received struct is: {'0': <tf.Tensor 'IteratorGetNext:1' shape=(128,) dtype=int64>, '1': <tf.Tensor 'IteratorGetNext:2' shape=(128,) dtype=int64>, '2': <tf.Tensor 'IteratorGetNext:13' shape=(128,) dtype=int64>, '3': <tf.Tensor 'IteratorGetNext:24' shape=(128,) dtype=int64>, '4': <tf.Tensor 'IteratorGetNext:35' shape=(128,) dtype=int64>, '5': <tf.Tensor 'IteratorGetNext:46' shape=(128,) dtype=int64>, '6': <tf.Tensor 'IteratorGetNext:57' shape=(128,) dtype=int64>, '7': <tf.Tensor 'IteratorGetNext:68' shape=(128,) dtype=int64>, '8': <tf.Tensor 'IteratorGetNext:79' shape=(128,) dtype=int64>, '9': <tf.Tensor 'IteratorGetNext:90' shape=(128,) dtype=int64>, '10': <tf.Tensor 'IteratorGetNext:3' shape=(128,) dtype=int64>, '11': <tf.Tensor 'IteratorGetNext:4' shape=(128,) dtype=int64>, '12': <tf.Tensor 'IteratorGetNext:5' shape=(128,) dtype=int64>, '13': <tf.Tensor 'IteratorGetNext:6' shape=(128,) dtype=int64>, '14': <tf.Tensor 'IteratorGetNext:7' shape=(128,) dtype=int64>, '15': <tf.Tensor 'IteratorGetNext:8' shape=(128,) dtype=int64>, '16': <tf.Tensor 'IteratorGetNext:9' shape=(128,) dtype=int64>, '17': <tf.Tensor 'IteratorGetNext:10' shape=(128,) dtype=int64>, '18': <tf.Tensor 'IteratorGetNext:11' shape=(128,) dtype=int64>, '19': <tf.Tensor 'IteratorGetNext:12' shape=(128,) dtype=int64>, '20': <tf.Tensor 'IteratorGetNext:14' shape=(128,) dtype=int64>, '21': <tf.Tensor 'IteratorGetNext:15' shape=(128,) dtype=int64>, '22': <tf.Tensor 'IteratorGetNext:16' shape=(128,) dtype=int64>, '23': <tf.Tensor 'IteratorGetNext:17' shape=(128,) dtype=int64>, '24': <tf.Tensor 'IteratorGetNext:18' shape=(128,) dtype=int64>, '25': <tf.Tensor 'IteratorGetNext:19' shape=(128,) dtype=int64>, '26': <tf.Tensor 'IteratorGetNext:20' shape=(128,) dtype=int64>, '27': <tf.Tensor 'IteratorGetNext:21' shape=(128,) dtype=int64>, '28': <tf.Tensor 'IteratorGetNext:22' shape=(128,) dtype=int64>, '29': <tf.Tensor 'IteratorGetNext:23' shape=(128,) dtype=int64>, '30': <tf.Tensor 'IteratorGetNext:25' shape=(128,) dtype=int64>, '31': <tf.Tensor 'IteratorGetNext:26' shape=(128,) dtype=int64>, '32': <tf.Tensor 'IteratorGetNext:27' shape=(128,) dtype=int64>, '33': <tf.Tensor 'IteratorGetNext:28' shape=(128,) dtype=int64>, '34': <tf.Tensor 'IteratorGetNext:29' shape=(128,) dtype=int64>, '35': <tf.Tensor 'IteratorGetNext:30' shape=(128,) dtype=int64>, '36': <tf.Tensor 'IteratorGetNext:31' shape=(128,) dtype=int64>, '37': <tf.Tensor 'IteratorGetNext:32' shape=(128,) dtype=int64>, '38': <tf.Tensor 'IteratorGetNext:33' shape=(128,) dtype=int64>, '39': <tf.Tensor 'IteratorGetNext:34' shape=(128,) dtype=int64>, '40': <tf.Tensor 'IteratorGetNext:36' shape=(128,) dtype=int64>, '41': <tf.Tensor 'IteratorGetNext:37' shape=(128,) dtype=int64>, '42': <tf.Tensor 'IteratorGetNext:38' shape=(128,) dtype=int64>, '43': <tf.Tensor 'IteratorGetNext:39' shape=(128,) dtype=int64>, '44': <tf.Tensor 'IteratorGetNext:40' shape=(128,) dtype=int64>, '45': <tf.Tensor 'IteratorGetNext:41' shape=(128,) dtype=int64>, '46': <tf.Tensor 'IteratorGetNext:42' shape=(128,) dtype=int64>, '47': <tf.Tensor 'IteratorGetNext:43' shape=(128,) dtype=int64>, '48': <tf.Tensor 'IteratorGetNext:44' shape=(128,) dtype=int64>, '49': <tf.Tensor 'IteratorGetNext:45' shape=(128,) dtype=int64>, '50': <tf.Tensor 'IteratorGetNext:47' shape=(128,) dtype=int64>, '51': <tf.Tensor 'IteratorGetNext:48' shape=(128,) dtype=int64>, '52': <tf.Tensor 'IteratorGetNext:49' shape=(128,) dtype=int64>, '53': <tf.Tensor 'IteratorGetNext:50' shape=(128,) dtype=int64>, '54': <tf.Tensor 'IteratorGetNext:51' shape=(128,) dtype=int64>, '55': <tf.Tensor 'IteratorGetNext:52' shape=(128,) dtype=int64>, '56': <tf.Tensor 'IteratorGetNext:53' shape=(128,) dtype=int64>, '57': <tf.Tensor 'IteratorGetNext:54' shape=(128,) dtype=int64>, '58': <tf.Tensor 'IteratorGetNext:55' shape=(128,) dtype=int64>, '59': <tf.Tensor 'IteratorGetNext:56' shape=(128,) dtype=int64>, '60': <tf.Tensor 'IteratorGetNext:58' shape=(128,) dtype=int64>, '61': <tf.Tensor 'IteratorGetNext:59' shape=(128,)dtype=int64>, '62': <tf.Tensor 'IteratorGetNext:60' shape=(128,) dtype=int64>, '63': <tf.Tensor 'IteratorGetNext:61' shape=(128,) dtype=int64>, '64': <tf.Tensor 'IteratorGetNext:62' shape=(128,) dtype=int64>, '65': <tf.Tensor 'IteratorGetNext:63' shape=(128,) dtype=int64>, '66': <tf.Tensor 'IteratorGetNext:64' shape=(128,) dtype=int64>, '67': <tf.Tensor 'IteratorGetNext:65' shape=(128,) dtype=int64>, '68': <tf.Tensor 'IteratorGetNext:66' shape=(128,) dtype=int64>, '69': <tf.Tensor 'IteratorGetNext:67' shape=(128,) dtype=int64>, '70': <tf.Tensor 'IteratorGetNext:69' shape=(128,) dtype=int64>, '71': <tf.Tensor 'IteratorGetNext:70' shape=(128,) dtype=int64>, '72': <tf.Tensor 'IteratorGetNext:71' shape=(128,) dtype=int64>, '73': <tf.Tensor 'IteratorGetNext:72' shape=(128,) dtype=int64>, '74': <tf.Tensor 'IteratorGetNext:73' shape=(128,) dtype=int64>, '75': <tf.Tensor 'IteratorGetNext:74' shape=(128,) dtype=int64>, '76': <tf.Tensor 'IteratorGetNext:75' shape=(128,) dtype=int64>, '77': <tf.Tensor 'IteratorGetNext:76' shape=(128,) dtype=int64>, '78': <tf.Tensor 'IteratorGetNext:77' shape=(128,) dtype=int64>, '79': <tf.Tensor 'IteratorGetNext:78' shape=(128,) dtype=int64>, '80': <tf.Tensor 'IteratorGetNext:80' shape=(128,) dtype=int64>, '81': <tf.Tensor 'IteratorGetNext:81' shape=(128,) dtype=int64>, '82': <tf.Tensor 'IteratorGetNext:82' shape=(128,) dtype=int64>, '83': <tf.Tensor 'IteratorGetNext:83' shape=(128,) dtype=int64>, '84': <tf.Tensor 'IteratorGetNext:84' shape=(128,) dtype=int64>, '85': <tf.Tensor 'IteratorGetNext:85' shape=(128,) dtype=int64>, '86': <tf.Tensor 'IteratorGetNext:86' shape=(128,) dtype=int64>, '87': <tf.Tensor 'IteratorGetNext:87' shape=(128,) dtype=int64>, '88': <tf.Tensor 'IteratorGetNext:88' shape=(128,) dtype=int64>, '89': <tf.Tensor 'IteratorGetNext:89' shape=(128,) dtype=int64>, '90': <tf.Tensor 'IteratorGetNext:91' shape=(128,) dtype=int64>, '91': <tf.Tensor 'IteratorGetNext:92' shape=(128,) dtype=int64>, '92': <tf.Tensor 'IteratorGetNext:93' shape=(128,) dtype=int64>, '93': <tf.Tensor 'IteratorGetNext:94' shape=(128,) dtype=int64>, '94': <tf.Tensor 'IteratorGetNext:95' shape=(128,) dtype=int64>, '95': <tf.Tensor 'IteratorGetNext:96' shape=(128,) dtype=int64>, '96': <tf.Tensor 'IteratorGetNext:97' shape=(128,) dtype=int64>}.
我该如何解决这个问题？
鉴于此数据集（基于您问题中的信息）：
import pandas as pd
import numpy as np
df = pd.DataFrame(np.random.randint(2, size=(5, 97))).astype(int)
df['vectorizer_input'] = ['test hello world']*5
df.to_csv('test.csv', index=False)
您可以尝试这样的操作：
tfdataset = tf.data.experimental.make_csv_dataset(
'/content/test.csv', batch_size=1, shuffle=False, num_epochs=1) # Do not forget num_epochs
def preprocess(data):
labels = list(dict(filter(lambda x: x[0] != 'vectorizer_input', data.items())).values())
return data['vectorizer_input'], labels
tfdataset = tfdataset.map(preprocess)
vectorizer = tf.keras.layers.TextVectorization(max_tokens=4000, output_sequence_length=4)
vectorizer.adapt(tfdataset.map(lambda x, y: x))
model = tf.keras.Sequential(
[
vectorizer,
tf.keras.layers.Embedding(4000, 64),
tf.keras.layers.GlobalMaxPool1D(),
tf.keras.layers.Dense(97, activation='sigmoid')
]
)
model.compile(
optimizer='adam',
loss=tf.keras.losses.BinaryCrossentropy(),
metrics=[tf.keras.metrics.Accuracy(), tf.keras.metrics.Precision(), tf.keras.metrics.Recall()])
model.fit(tfdataset.batch(2), epochs=10)
我正在使用具有 1 个特征列（字符串）和 97 个标签列（多标签分类）的 CSV 数据集，每行 1 或 0。 数据：
| | vectorizer_input | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 |
|---:|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|
| 0 | want improve question update question on-topic stack overflow closed 8 year ago json need update {activity_code1 activity_namefootball} {activity_code1 activity_nametennis} based activity_code achieve php first need decode change data re-encode save back file | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 1 | string xml need return xml document default returned content type text plain content rendered need content type application xml enabled option respectbrowseracceptheader serialize object xml set correct content type except object string string xml need return xml document return contentresult controller startup project.json response github good measure | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 2 | volumetric imaging data consisting value sampled regular grid x z non-cubic voxel shape space adjacent point z greater x would eventually like able interpolate value arbitrary 2d plane pass volume like aware scipy.ndimage.map_coordinates case using le straightforward implicitly assumes spacing element input array equal across dimension could first resample input array according smallest voxel dimension voxels would cube use map_coordinates interpolate plane seem like great idea interpolate data twice also aware scipy various interpolators irregularly-spaced nd data linearndinterpolator nearestndinterpolator etc. slow memory-intensive purpose best way interpolating data given know value regularly spaced within dimension use map_coordinates little bit algebra let say spacing grid dx dy dz need map real world coordinate array index coordinate let define three new variable array index input map_coordinates array shape ... number dimension original data define array transform real world coordinate array index coordinate dividing scaling little broadcasting magic put together example let say want interpolate value along plane 2*y - z = 0 take two vector perpendicular plane normal vector get coordinate want interpolate convert array index coordinate interpoalte using map_coordinates last array shape 10 10 position [u_idx v_idx] value corresponding coordinate coords[ u_idx v_idx] could build idea handle interpolation coordinate start zero adding offset scaling | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 3 | best way calculate sum matrix a^i + a^ i+1 + a^i+2........a^n large n thought two possible way 1 use logarithmic matrix exponentiation lme a^i calculate subsequent matrix multiplying problem really take advantage lme algorithm using lowest power 2 use lme finding a^n memoize intermediate calculation problem much space required large n third way notice let logarithmic number step need compute inverse direct implementation lead log n ^2 factor keep log n computing power compute b | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 4 | named color available matplotlib use plot find list matplotlib documentation claim name however found color also used least context list anyone know exhaustive list named color available constantly forget name color want use keep coming back question = previous answer great find bit difficult get overview available color posted image prefer color grouped similar color slightly tweaked matplotlib answer mentioned comment get color list sorted column order identical would sort eye think give good overview updated image code reflect arebeccapurple added three sage color moved xkcd prefix since posted answer originally really change much matplotlib example code completeness updated 2017-10-25 merged previous update section would like use additional named color plotting matplotlib use xkcd crowdsourced color name via xkcd prefix access plethora named color default tableau color available matplotlib via notab prefix ten distinct color also plot color html hex code similar specifying rgb tuple rather named color apart fact hex code passed string include image 16 million color choose from.. detail please refer matplotlib color documentation source file specifying available color _color_data.py | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
进口：
import re
import tensorflow as tf
from nltk.corpus import stopwords
from datasets import load_dataset
from nltk.stem import WordNetLemmatizer
import numpy as np
from tensorflow.keras import models, layers, losses, metrics
from sklearn.metrics import classification_report
我如何加载数据集：
feature_cols = ['vectorizer_input']
label_cols = [str(i) for i in range(97)]
tfdataset = tf.data.experimental.make_csv_dataset(
name,
batch_size=128,
column_names=feature_cols + label_cols,
column_defaults=[tf.dtypes.string] + [tf.dtypes.int64] * 97,
shuffle=False
)
然后我指定多个标签列：
def make_features_labels_tuple(x, feature_cols, label_cols):
labels = {k: x[k] for k in x if k in label_cols}
features = {k: x[k] for k in x if k in feature_cols}
return features, labels
tfdataset = tfdataset.map(lambda x: make_features_labels_tuple(x, feature_cols, label_cols))
这是我现在的简单模型：
vectorizer = layers.TextVectorization(max_tokens=4000, output_sequence_length=4000)
model = models.Sequential(
[
vectorizer,
layers.Embedding(4000, 64),
layers.GlobalMaxPool1D(),
layers.Dense(97, activation='sigmoid')
]
)
model.compile(
optimizer='adam',
loss=losses.BinaryCrossentropy(),
metrics=[metrics.Accuracy(), metrics.Precision(), metrics.Recall()]
)
模型摘要：
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
text_vectorization_44 (Text (None, 4000) 0
Vectorization)
embedding_16 (Embedding) (None, 4000, 64) 256000
global_max_pooling1d_16 (Gl (None, 64) 0
obalMaxPooling1D)
dense_16 (Dense) (None, 97) 6305
=================================================================
Total params: 262,305
Trainable params: 262,305
Non-trainable params: 0
_________________________________________________________________
这是我在尝试 model.fit:
ValueError: Found unexpected losses or metrics that do not correspond to any Model output: dict_keys(['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '20', '21', '22', '23', '24', '25', '26', '27', '28', '29', '30', '31', '32', '33', '34', '35', '36', '37', '38', '39', '40', '41', '42', '43', '44', '45', '46', '47', '48', '49', '50', '51', '52', '53', '54', '55', '56', '57', '58', '59', '60', '61', '62', '63', '64', '65', '66', '67', '68', '69', '70', '71', '72', '73', '74', '75', '76', '77', '78', '79', '80', '81', '82', '83', '84', '85', '86', '87', '88', '89', '90', '91', '92', '93', '94', '95', '96']). Valid mode output names: ['dense_7']. Received struct is: {'0': <tf.Tensor 'IteratorGetNext:1' shape=(128,) dtype=int64>, '1': <tf.Tensor 'IteratorGetNext:2' shape=(128,) dtype=int64>, '2': <tf.Tensor 'IteratorGetNext:13' shape=(128,) dtype=int64>, '3': <tf.Tensor 'IteratorGetNext:24' shape=(128,) dtype=int64>, '4': <tf.Tensor 'IteratorGetNext:35' shape=(128,) dtype=int64>, '5': <tf.Tensor 'IteratorGetNext:46' shape=(128,) dtype=int64>, '6': <tf.Tensor 'IteratorGetNext:57' shape=(128,) dtype=int64>, '7': <tf.Tensor 'IteratorGetNext:68' shape=(128,) dtype=int64>, '8': <tf.Tensor 'IteratorGetNext:79' shape=(128,) dtype=int64>, '9': <tf.Tensor 'IteratorGetNext:90' shape=(128,) dtype=int64>, '10': <tf.Tensor 'IteratorGetNext:3' shape=(128,) dtype=int64>, '11': <tf.Tensor 'IteratorGetNext:4' shape=(128,) dtype=int64>, '12': <tf.Tensor 'IteratorGetNext:5' shape=(128,) dtype=int64>, '13': <tf.Tensor 'IteratorGetNext:6' shape=(128,) dtype=int64>, '14': <tf.Tensor 'IteratorGetNext:7' shape=(128,) dtype=int64>, '15': <tf.Tensor 'IteratorGetNext:8' shape=(128,) dtype=int64>, '16': <tf.Tensor 'IteratorGetNext:9' shape=(128,) dtype=int64>, '17': <tf.Tensor 'IteratorGetNext:10' shape=(128,) dtype=int64>, '18': <tf.Tensor 'IteratorGetNext:11' shape=(128,) dtype=int64>, '19': <tf.Tensor 'IteratorGetNext:12' shape=(128,) dtype=int64>, '20': <tf.Tensor 'IteratorGetNext:14' shape=(128,) dtype=int64>, '21': <tf.Tensor 'IteratorGetNext:15' shape=(128,) dtype=int64>, '22': <tf.Tensor 'IteratorGetNext:16' shape=(128,) dtype=int64>, '23': <tf.Tensor 'IteratorGetNext:17' shape=(128,) dtype=int64>, '24': <tf.Tensor 'IteratorGetNext:18' shape=(128,) dtype=int64>, '25': <tf.Tensor 'IteratorGetNext:19' shape=(128,) dtype=int64>, '26': <tf.Tensor 'IteratorGetNext:20' shape=(128,) dtype=int64>, '27': <tf.Tensor 'IteratorGetNext:21' shape=(128,) dtype=int64>, '28': <tf.Tensor 'IteratorGetNext:22' shape=(128,) dtype=int64>, '29': <tf.Tensor 'IteratorGetNext:23' shape=(128,) dtype=int64>, '30': <tf.Tensor 'IteratorGetNext:25' shape=(128,) dtype=int64>, '31': <tf.Tensor 'IteratorGetNext:26' shape=(128,) dtype=int64>, '32': <tf.Tensor 'IteratorGetNext:27' shape=(128,) dtype=int64>, '33': <tf.Tensor 'IteratorGetNext:28' shape=(128,) dtype=int64>, '34': <tf.Tensor 'IteratorGetNext:29' shape=(128,) dtype=int64>, '35': <tf.Tensor 'IteratorGetNext:30' shape=(128,) dtype=int64>, '36': <tf.Tensor 'IteratorGetNext:31' shape=(128,) dtype=int64>, '37': <tf.Tensor 'IteratorGetNext:32' shape=(128,) dtype=int64>, '38': <tf.Tensor 'IteratorGetNext:33' shape=(128,) dtype=int64>, '39': <tf.Tensor 'IteratorGetNext:34' shape=(128,) dtype=int64>, '40': <tf.Tensor 'IteratorGetNext:36' shape=(128,) dtype=int64>, '41': <tf.Tensor 'IteratorGetNext:37' shape=(128,) dtype=int64>, '42': <tf.Tensor 'IteratorGetNext:38' shape=(128,) dtype=int64>, '43': <tf.Tensor 'IteratorGetNext:39' shape=(128,) dtype=int64>, '44': <tf.Tensor 'IteratorGetNext:40' shape=(128,) dtype=int64>, '45': <tf.Tensor 'IteratorGetNext:41' shape=(128,) dtype=int64>, '46': <tf.Tensor 'IteratorGetNext:42' shape=(128,) dtype=int64>, '47': <tf.Tensor 'IteratorGetNext:43' shape=(128,) dtype=int64>, '48': <tf.Tensor 'IteratorGetNext:44' shape=(128,) dtype=int64>, '49': <tf.Tensor 'IteratorGetNext:45' shape=(128,) dtype=int64>, '50': <tf.Tensor 'IteratorGetNext:47' shape=(128,) dtype=int64>, '51': <tf.Tensor 'IteratorGetNext:48' shape=(128,) dtype=int64>, '52': <tf.Tensor 'IteratorGetNext:49' shape=(128,) dtype=int64>, '53': <tf.Tensor 'IteratorGetNext:50' shape=(128,) dtype=int64>, '54': <tf.Tensor 'IteratorGetNext:51' shape=(128,) dtype=int64>, '55': <tf.Tensor 'IteratorGetNext:52' shape=(128,) dtype=int64>, '56': <tf.Tensor 'IteratorGetNext:53' shape=(128,) dtype=int64>, '57': <tf.Tensor 'IteratorGetNext:54' shape=(128,) dtype=int64>, '58': <tf.Tensor 'IteratorGetNext:55' shape=(128,) dtype=int64>, '59': <tf.Tensor 'IteratorGetNext:56' shape=(128,) dtype=int64>, '60': <tf.Tensor 'IteratorGetNext:58' shape=(128,) dtype=int64>, '61': <tf.Tensor 'IteratorGetNext:59' shape=(128,)dtype=int64>, '62': <tf.Tensor 'IteratorGetNext:60' shape=(128,) dtype=int64>, '63': <tf.Tensor 'IteratorGetNext:61' shape=(128,) dtype=int64>, '64': <tf.Tensor 'IteratorGetNext:62' shape=(128,) dtype=int64>, '65': <tf.Tensor 'IteratorGetNext:63' shape=(128,) dtype=int64>, '66': <tf.Tensor 'IteratorGetNext:64' shape=(128,) dtype=int64>, '67': <tf.Tensor 'IteratorGetNext:65' shape=(128,) dtype=int64>, '68': <tf.Tensor 'IteratorGetNext:66' shape=(128,) dtype=int64>, '69': <tf.Tensor 'IteratorGetNext:67' shape=(128,) dtype=int64>, '70': <tf.Tensor 'IteratorGetNext:69' shape=(128,) dtype=int64>, '71': <tf.Tensor 'IteratorGetNext:70' shape=(128,) dtype=int64>, '72': <tf.Tensor 'IteratorGetNext:71' shape=(128,) dtype=int64>, '73': <tf.Tensor 'IteratorGetNext:72' shape=(128,) dtype=int64>, '74': <tf.Tensor 'IteratorGetNext:73' shape=(128,) dtype=int64>, '75': <tf.Tensor 'IteratorGetNext:74' shape=(128,) dtype=int64>, '76': <tf.Tensor 'IteratorGetNext:75' shape=(128,) dtype=int64>, '77': <tf.Tensor 'IteratorGetNext:76' shape=(128,) dtype=int64>, '78': <tf.Tensor 'IteratorGetNext:77' shape=(128,) dtype=int64>, '79': <tf.Tensor 'IteratorGetNext:78' shape=(128,) dtype=int64>, '80': <tf.Tensor 'IteratorGetNext:80' shape=(128,) dtype=int64>, '81': <tf.Tensor 'IteratorGetNext:81' shape=(128,) dtype=int64>, '82': <tf.Tensor 'IteratorGetNext:82' shape=(128,) dtype=int64>, '83': <tf.Tensor 'IteratorGetNext:83' shape=(128,) dtype=int64>, '84': <tf.Tensor 'IteratorGetNext:84' shape=(128,) dtype=int64>, '85': <tf.Tensor 'IteratorGetNext:85' shape=(128,) dtype=int64>, '86': <tf.Tensor 'IteratorGetNext:86' shape=(128,) dtype=int64>, '87': <tf.Tensor 'IteratorGetNext:87' shape=(128,) dtype=int64>, '88': <tf.Tensor 'IteratorGetNext:88' shape=(128,) dtype=int64>, '89': <tf.Tensor 'IteratorGetNext:89' shape=(128,) dtype=int64>, '90': <tf.Tensor 'IteratorGetNext:91' shape=(128,) dtype=int64>, '91': <tf.Tensor 'IteratorGetNext:92' shape=(128,) dtype=int64>, '92': <tf.Tensor 'IteratorGetNext:93' shape=(128,) dtype=int64>, '93': <tf.Tensor 'IteratorGetNext:94' shape=(128,) dtype=int64>, '94': <tf.Tensor 'IteratorGetNext:95' shape=(128,) dtype=int64>, '95': <tf.Tensor 'IteratorGetNext:96' shape=(128,) dtype=int64>, '96': <tf.Tensor 'IteratorGetNext:97' shape=(128,) dtype=int64>}.
我该如何解决这个问题？
鉴于此数据集（基于您问题中的信息）：
import pandas as pd
import numpy as np
df = pd.DataFrame(np.random.randint(2, size=(5, 97))).astype(int)
df['vectorizer_input'] = ['test hello world']*5
df.to_csv('test.csv', index=False)
您可以尝试这样的操作：
tfdataset = tf.data.experimental.make_csv_dataset(
'/content/test.csv', batch_size=1, shuffle=False, num_epochs=1) # Do not forget num_epochs
def preprocess(data):
labels = list(dict(filter(lambda x: x[0] != 'vectorizer_input', data.items())).values())
return data['vectorizer_input'], labels
tfdataset = tfdataset.map(preprocess)
vectorizer = tf.keras.layers.TextVectorization(max_tokens=4000, output_sequence_length=4)
vectorizer.adapt(tfdataset.map(lambda x, y: x))
model = tf.keras.Sequential(
[
vectorizer,
tf.keras.layers.Embedding(4000, 64),
tf.keras.layers.GlobalMaxPool1D(),
tf.keras.layers.Dense(97, activation='sigmoid')
]
)
model.compile(
optimizer='adam',
loss=tf.keras.losses.BinaryCrossentropy(),
metrics=[tf.keras.metrics.Accuracy(), tf.keras.metrics.Precision(), tf.keras.metrics.Recall()])
model.fit(tfdataset.batch(2), epochs=10)