如何提高Google Cloud Vision PDF文本检测20页的限制？

Question

您好，我正在尝试使用 Google Cloud Vision 检测 34 页 PDF 中的文本。然后，我将生成的 JSON 中的文本保存到 Firebase Firestore，并保存到我的 Firebase 存储中。所有这一切都使用 Firebase Cloud 功能。当我循环遍历 JSON 中的页面时，我只得到 20 页，而不是全部 34 页。我使用的任何超过 20 页的 PDF 都会发生同样的情况。我使用 asyncBatchAnnotateFiles 函数，该函数应该有 2,000 页限制。我到处搜索以了解如何增加 20 页的限制，但未能找到任何相关信息。请帮忙。这是我的代码：

if (contentType === "application/pdf") {  

    const orderUri = path.dirname(object.name);
    const fileName = path.basename(object.name)
    const outputPrefix = 'results';
    const gcsSourceUri = `gs://${object.bucket}/${object.name}`;
    const gcsDestinationUri = `gs://${object.bucket}/${orderUri}/${outputPrefix}/${fileName}/`;
    

    const inputConfig = {
    // Supported mime_types are: 'application/pdf' and 'image/tiff'
    mimeType: 'application/pdf',
    gcsSource: {
        uri: gcsSourceUri,
    },
    };

    const outputConfig = {
    gcsDestination: {
        uri: gcsDestinationUri,
    },
    };

    const features = [{type: 'DOCUMENT_TEXT_DETECTION'}];

    const request = {
    requests: [
        {
        inputConfig: inputConfig,
        features: features,
        outputConfig: outputConfig,
        },
    ],
    };

    // OCR PDFs
    const [operation] = await client.asyncBatchAnnotateFiles(request);
    const [filesResponse] = await operation.promise();    
    // const destinationUri =
    // filesResponse.responses[0].outputConfig.gcsDestination.uri;
    // console.log('Json saved to: ' + destinationUri);    
    
    
}

以下是我如何从 Firebase Storage 检索 JSON 并循环遍历它以获取文本：

if (path.basename(object.name).startsWith('output') && path.basename(object.name).split('.').pop() === "json") {       
        // Get references
        const fileBucket = object.bucket; // The Storage bucket that contains the file.
        const filePath = object.name; // File path in the bucket.       

         // Download JSON     
        const bucket = admin.storage().bucket(fileBucket);   
        const downloadResponse = await bucket.file(filePath).download();       
        const bufferToJson = downloadResponse.toString();
        const jsObject = JSON.parse(bufferToJson);

        // Text
        const textArray = jsObject.responses.map(async (response) => {
          return response.fullTextAnnotation.text;
        });
        const readyArray = await Promise.all(textArray);
        const fullTextReady = readyArray.join();

        // Count words
        function countWords(str) {
          return str.trim().split(/\s+/).length;
        }
        const words = countWords(fullTextReady);       


        // Text confidence
        const textConfidenceArray = jsObject.responses.map(async (response) => {
          return response.fullTextAnnotation.pages.map((page) => {
            return page.confidence;
          })
        })
        const textConfidence = await Promise.all(textConfidenceArray);        
        const textConfidence2 = textConfidence.flat();
        const sum = textConfidence2.reduce((accumulator, currentValue) => {
          return accumulator + currentValue
        },0);
        const average = sum / textConfidence2.length;
        const textConfidence3 = Number(average).toFixed(2) * 100;      
        
        
        // Language and Language Confidence
        const pages = jsObject.responses.map((response) => {
          return response.fullTextAnnotation.pages.map((page) => {
            return page.property.detectedLanguages
          })
        });
        const pages2 = await Promise.all(pages);
        const detectedLanguages = pages2.flat(2);    

        const languageAndConfidenceArray = detectedLanguages.map((language) => {
               const langCode = language.languageCode;
               const confidence = Number((language.confidence).toFixed(1)) * 100;
                return {
                  languageCode: langCode,
                  languageConfidence: confidence
                }
        })

        const languages = await Promise.all(languageAndConfidenceArray);

             
        // Save to Firestore
        const jsonLocation = path.dirname(object.name);
        const fileName = path.basename(jsonLocation);
        const results = path.dirname(jsonLocation);
        const order = path.dirname(results);   
        const destination = `${order}/${fileName}`;
        const docRef = db.collection('Clients').doc(destination);
        await docRef.set({
          fullText: fullTextReady,
          textConfidence: textConfidence3,         
          type: "application/pdf",
          pageCount: jsObject.responses.length,
          languages: languages,
          fileName: fileName,
          location: jsonLocation,
          wordCount: words          
          }, { merge: true });
            
}

Answer 1

经过长时间的研究，我终于弄清楚了。您需要像这样更改输出配置中的批量大小：

JSON representation

{
  "gcsDestination": {
    object (GcsDestination)
  },
  "batchSize": integer
}

放入 Google Cloud Storage 上每个输出 JSON 文件的响应原型的最大数量。有效范围为 [1, 100]。如果未指定，则默认值为 20。

这里是官方文档的链接：https://cloud.google.com/vision/docs/reference/rest/v1/OutputConfig