使用selenium发送带有有效负载的请求时出现空指针异常

Question

我正在尝试使用硒绕过Cloud Flare Protection。 首先，我向主页发送了一个请求，成功绕过了 CloudFlare， 然后从家里我得到 ViewState，ViewStateGuid 等，然后再次使用有效负载发送请求，此时我收到一个错误，如下所示

这是我的代码

    val url = "https:abc.com"
    val searchPageUrl = "abc.com?

  System.setProperty("webdriver.chrome.driver", "/home/asif/Downloads/chromedriver_linux64/chromedriver")
  val chromeOptions = new ChromeOptions()
  chromeOptions.addArguments("--remote-allow-origins=*")
  chromeOptions.setCapability("javascriptEnabled", true)
  chromeOptions.setCapability("acceptSslCerts", true)

  val driver = new ChromeDriver(chromeOptions)

  driver.get(url)

  val html = driver.getPageSource
  val browser = new JsoupBrowser()
  val homePageDoc = browser.parseString(html)
  val __VIEWSTATEGUID = homePageDoc >> element("input[name='__VIEWSTATEGUID']") >> attr("value")
  val __EVENTTARGET = homePageDoc >> element("input[name='__EVENTTARGET']") >> attr("value")
  val ScriptManager1_TSM = homePageDoc >> element("input[name='ScriptManager1_TSM']") >> attr("value")
  val __LASTFOCUS = homePageDoc >> element("input[name='__LASTFOCUS']") >> attr("value")
  val __VIEWSTATE = homePageDoc >> element("input[name='__VIEWSTATE']") >> attr("value")
  val __EVENTARGUMENT = homePageDoc >> element("input[name='__EVENTARGUMENT']") >> attr("value")

  val formData = Map(
    "ScriptManager1_TSM" -> ScriptManager1_TSM,
    "__EVENTTARGET" -> __EVENTTARGET,
    "__EVENTARGUMENT" -> __EVENTARGUMENT,
    "__LASTFOCUS" -> __LASTFOCUS,
    "__VIEWSTATEGUID" -> __VIEWSTATEGUID,
    "__VIEWSTATE" -> __VIEWSTATE,
    "__SCROLLPOSITIONX" -> "12",
    "__SCROLLPOSITIONY" -> "310",
    "ctl00$MainContent$LastNameTextBox" -> "eli",
    "ctl00$MainContent$FirstNameTextBox" -> "",
    "ctl00$MainContent$MiddleInitialTextBox" -> "",
    "ctl00$MainContent$NotaryIdTextBox" -> "",
    "ctl00$MainContent$BusinessNameTextBox" -> "",
    "ctl00$MainContent$ZipCodeTextBox" -> "",
    "ctl00$MainContent$CountyCombo" -> "",
    "ctl00$MainContent$ActiveNotariesCheckBox" -> "",
    "ctl00$MainContent$SearchButton" -> "Search"
  )

  val formDataString = formData.map { case (key, value) => s"$key=${java.net.URLEncoder.encode(value, "UTF-8")}" }.mkString("&")

  val postBody = formDataString

  val searchPageDriver = new ChromeDriver(chromeOptions)

  searchPageDriver.get(searchPageUrl + postBody)

  val html1 = searchPageDriver.getPageSource
  val doc = browser.parseString(html1)

  println(doc)

  driver.close()
  searchPageDriver.close()

错误是：

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

[ArgumentNullException: Value cannot be null.
Parameter name: inputString]
   System.Web.UI.ObjectStateFormatter.Deserialize(String inputString, Purpose purpose) +684
   System.Web.UI.ObjectStateFormatter.System.Web.UI.IStateFormatter.Deserialize(String serializedState) +49
   DatabasePageStatePersister.Load() in \\ad.local\shares\Web\Webdata\websites\apps.azsos.gov\apps\notary\search\App_Code\DatabasePageStatePersister.cs:64
   System.Web.UI.Page.LoadPageStateFromPersistenceMedium() +58
   System.Web.UI.Page.LoadAllState() +46
   System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +859

Answer 1

val __VIEWSTATEGUID = homePageDoc >> element("input[name='__VIEWSTATEGUID']") >> attr("value")
  val __EVENTTARGET = homePageDoc >> element("input[name='__EVENTTARGET']") >> attr("value")
  val ScriptManager1_TSM = homePageDoc >> element("input[name='ScriptManager1_TSM']") >> attr("value")
  val __LASTFOCUS = homePageDoc >> element("input[name='__LASTFOCUS']") >> attr("value")
  val __VIEWSTATE = homePageDoc >> element("input[name='__VIEWSTATE']") >> attr("value")
  val __EVENTARGUMENT = homePageDoc >> element("input[name='__EVENTARGUMENT']") >> attr("value")

其中一些锚点找不到元素，因此返回 null，然后您的代码在进行地图处理的行上失败。