我正在使用管道作业通过 GCP 上的 Vertex AI 微调文本野牛模型。我的 API 使用专用服务帐户进行记录。它具有以下角色:Vertex AI 服务代理和服务帐户用户。
我正在使用这个 Node.js 代码:
import aiplatform from '@google-cloud/aiplatform';
class ModelTuningController {
initiateModelTuning = catchAsync(async (req, res) => {
const { modelDisplayName = 'fineTunedTest' } = req.body;
const { PipelineServiceClient } = aiplatform.v1;
const location = 'us-central1';
const project = 'projectID';
const model = 'text-bison@001';
const pipelineClient = new PipelineServiceClient({
apiEndpoint: `${location}-aiplatform.googleapis.com`,
keyFilename: config.vector_store_key_filename,
});
const parameters = {
source_model: { stringValue: model },
train_dataset: {
stringValue: 'gs://path/file.jsonl',
},
tuned_model_display_name: {
stringValue: modelDisplayName,
},
epochs: {
numberValue: 4,
},
learning_rate_multiplier: {
numberValue: 1,
},
};
const runtimeConfig = {
gcsOutputDirectory: 'gs://output-path',
parameterValues: parameters,
};
try {
const parent = `projects/${project}/locations/${location}`;
const pipelineJob = {
displayName: modelDisplayName,
runtimeConfig,
};
const request = {
parent,
pipelineJob,
};
const [response] = await pipelineClient.createPipelineJob(request);
return sendResponse(res, {
message: ModelTuningMessages.jobInitiated(response.name),
});
} catch (err) {
return sendError(
err,
new ApiError(
httpStatus.INTERNAL_SERVER_ERROR,
ModelTuningMessages.jobError
)
);
}
});
}
export default ModelTuningController;
我遇到以下错误:您无权充当 service_account:${projectID}[email protected]。 (或者它可能不存在)。
问题是 ${projectID}[email protected] 是项目默认服务帐户。我猜我的服务帐户应该充当项目的默认服务帐户。
我的服务帐户是否缺少在 Vertex AI 上执行微调作业的角色?
___________
编辑
看来我需要激活计算引擎 API 才能拥有默认的计算引擎服务帐户才能使其正常工作!
但是我有错误:
Error: 3 INVALID_ARGUMENT:
。很难知道哪个论证是无效的。我发送到管道作业的请求如下:
{
"parent": "projects/${projectID}/locations/us-central1",
"pipelineJob": {
"displayName": "fineTunedTest",
"runtimeConfig": {
"gcsOutputDirectory": "gs://${output}",
"parameterValues": {
"source_model": { "stringValue": "gemini-1.0-pro-002" },
"train_dataset": { "stringValue": "gs://${input}/${file}" },
"tuned_model_display_name": { "stringValue": "fineTunedTest" },
"epochs": { "numberValue": 4 },
"learning_rate_multiplier": { "numberValue": 1 }
}
}
}
}
问题在于未传递适当的服务帐户凭证并授予适当的 IAM 权限。有两种解决方案(首选第二种):
${projectID}[email protected]
。要运行 Vertex 管道,默认服务帐户需要具有以下 IAM 权限:vertex AI user
角色授予默认服务帐户。import aiplatform from '@google-cloud/aiplatform';
import { Auth, google } from "googleapis";
class ModelTuningController {
initiateModelTuning = catchAsync(async (req, res) => {
const { modelDisplayName = 'fineTunedTest' } = req.body;
const { PipelineServiceClient } = aiplatform.v1;
# set up auth
// Path to your service account key file
const keyFilename = '/path/to/your/service-account.json';
// Create a new GoogleAuth instance with the service account key file
const auth = new Auth.GoogleAuth({
keyFile: keyFilename,
scopes: 'https://www.googleapis.com/auth/cloud-platform'
});
// Get credentials
const client = await auth.getClient();
const clientOptions = {
apiEndpoint: '${location}-aiplatform.googleapis.com',
auth: client // Pass the authenticated client object here
};
const location = 'us-central1';
const project = 'projectID';
const model = 'text-bison@001';
const pipelineClient = new PipelineServiceClient(clientOptions);
...
请记住,首选身份验证方法取决于运行管道 Node.js 代码的位置和方式。即本地运行与在 GCP 云功能中运行。理想情况下,您不想导出服务帐户密钥文件以提高安全性。