0

I'm trying to use a list of variables to run to get data from an IBM BigFix Relay Diagnostic Page, currently using defined variables I can run the following to get what I need:

def fs_grab(urllookup):
    try:
        html = requests.get(urllookup, timeout=3).text
        fs_start = 'FillDB File Size Limit:</div><div class="forminput">'
        fs_end = '% ( '
        fs = html.split(fs_start)[-1].split(fs_end)[0]
        fs = float(fs)
        return fs
    except Exception:
        logging.error('Timeout', exc_info=True)
        pass

How can I get the fs value from a list of variables and append to the list? Eventually I would like to insert them into a MySQL Database, but for now, just trying one step at a time and append it to the existing list if possible.

Raw markup:

<div class="formline"><div class="formlabel">FillDB File Size Limit:</div><div class="formlabel">FillDB File Size Limit:</div><div class="forminput">0.0% ( 0 / 826634880 Bytes )</div></div><div class="formline"><div class="formlabel">FillDB File Count Limit:</div><div class="forminput">0.06% ( 6 / 10000 Files )</div></div><div class="formline"><div class="formlabel">FillDB BigFix Query File Size Limit:</div><div class="forminput">0.0% ( 0 / 3145728 Bytes )</div>

Pretty printed

<div class="formline">
    <div class="formlabel">FillDB File Size Limit:</div>
    <div class="formlabel">FillDB File Size Limit:</div>
    <div class="forminput">0.0% ( 0 / 826634880 Bytes )</div>
</div>
<div class="formline">
    <div class="formlabel">FillDB File Count Limit:</div>
    <div class="forminput">0.06% ( 6 / 10000 Files )</div>
</div>
<div class="formline">
    <div class="formlabel">FillDB BigFix Query File Size Limit:</div>
    <div class="forminput">0.0% ( 0 / 3145728 Bytes )</div>
</div>

The HTML of the page is above, I'm really only looking for the numerical values of the File Size and File Count

wwii
  • 23,232
  • 7
  • 37
  • 77
sdhummel
  • 15
  • 4
  • can you give example of what you are trying to achieve? perhaps example of HTML and what you are trying to extract. – Pratibha Jun 23 '20 at 01:08
  • `get the fs value from a list` - which list? `append it to the existing list` - which list? – wwii Jun 23 '20 at 02:00
  • I'm trying to pull in a list of servers from an external file, then run through that list using requests and get the data. So the list would be like server1 server2 server3 And for requests to work it would have to go to http://server01 and get a value from the page, go to http://server02, http://server03, etc – sdhummel Jun 23 '20 at 12:37
  • FillDB File Size Limit:
    0.0% ( 0 / 826634880 Bytes )
    FillDB File Count Limit:
    0.06% ( 6 / 10000 Files )
    FillDB BigFix Query File Size Limit:
    0.0% ( 0 / 3145728 Bytes ) The HTML of the page is above, I'm really only looking for the numerical values of the File Size and File Count
     – sdhummel Jun 23 '20 at 12:41
  • Do you have objections to using a package that did not come *built-in* with Python? – wwii Jun 23 '20 at 13:54
  • Not at all, I'm trying to make the code as efficient as possible since it has to process through about 400 servers. – sdhummel Jun 23 '20 at 13:57
  • Your function works? Are you just asking how to iterate over a sequence of url's' pass each one to the function, and accumulate the results? – wwii Jun 23 '20 at 14:02
  • For future reference: [Formatting help](https://stackoverflow.com/editing-help)... [Formatting sandbox](https://meta.stackexchange.com/questions/3122/formatting-sandbox). Welcome to SO make sure you take the time to take the [tour] and read [ask] and the other links found on that page. – wwii Jun 23 '20 at 14:04
  • Everything works as it is now, the problem is I'm currently hardcoding variables, which with only 20 servers to check isn't a big deal, but now that it's going to be over 400 servers I don't want to hard assign 400 variables in the script. Right now I have 2 variables per server set, fc_server1 and fs_server1 for File Count and Size, and then that gets inputted into a database and uses Grafana as a visual front-end. – sdhummel Jun 23 '20 at 14:09
  • 2
    Does this answer your question? [How do I create a variable number of variables?](https://stackoverflow.com/questions/1373164/how-do-i-create-a-variable-number-of-variables). Your question title and the example you gave don't appear to reflect the problem you are trying to solve. Also: [How can you dynamically create variables via a while loop?](https://stackoverflow.com/questions/5036700/how-can-you-dynamically-create-variables-via-a-while-loop). – wwii Jun 23 '20 at 14:35
  • I'd say the consensus from those two would be to use a single dictionary with the keys being the server urls (string formatted) and the values being a list or dict of the stuff you want. Though you might want to use a list of dicts - `[{'url':...,'fs':...,'fc':...},{...}]` or a list of [namedtuples](https://docs.python.org/3/library/collections.html#collections.namedtuple) constructed like the list of dicts I mentioned. All of which can be *constructed on-the-fly*. – wwii Jun 23 '20 at 14:40

0 Answers0