New activity vs recent posts results and SMS vs MMS

[micjgam]

Is there a way to discern and code such that the scraper will run according to recent posts as opposed to new activity? I would like to see the results coming in chronologically.

Also, is there a way to remove emoticons, etc. from a post so that all I am left with is an SMS message that is texted to myself?

[neon-ninja]

The posts come in the order that Facebook serves them. You can sort them by time yourself (something like sorted(posts, key=lambda p: p["time"])). Perhaps try check the differences with and without cookies, as sometimes without cookies means post order isn't targeted to your account.

Yes, it's very easy to remove emoticons. You can remove everything that isn't ASCII with this - post["text"].encode('ascii', 'ignore').decode('ascii')

[micjgam]

Thank you again... the encode.decode method was not working for me last night (found this solution elsewhere), but I got it working.

The sorted function is taking some doing. Currently getting "TypeError: string indices must be integers". Going to look a little more into the lambda function.

for post in get_posts(group=GROUP ID, credentials=(email, password), pages=5):
    keyword = KEYWORD
    if keyword in post['post_text']:
        sorted(post, key=lambda x: x['time'])
        date = str(post['time'])
        posting = reft(post['post_text'][:75])
        poster = post['username']
        sms.send(f'post {date}\r\r{poster}\r\r{posting}', PHONE NUMBER)

[neon-ninja]

No, you need to consume the generator before sorting. Like so:

posts = list(get_posts(group=GROUP ID, credentials=(email, password), pages=5))
posts = sorted(posts, key=lambda x: x["time"])
for post in posts:
    keyword = KEYWORD
    if keyword in post['post_text']:
        date = str(post['time'])
        posting = reft(post['post_text'][:75])
        poster = post['username']
        sms.send(f'post {date}\r\r{poster}\r\r{posting}', PHONE NUMBER)

[micjgam]

Ran the code and got back to back IP bans. Is this because it's calling the sort (thereby performing an initial scrape) and then iterating to scrape through the posts a second time?

[neon-ninja]

No. Maybe try cookies instead of credentials

[micjgam]

It worked.

So, do I only need the c_user and xs values in the JSON file? If so, then I would pair it down to these only.

I am getting a PytzUsageWarning (I was getting this a week ago, but it went away). Is this from the expirationDate values in the cookies file?

I am going to Dockerize this small app. Would I still need to secure the cookies values as I would have for the credentials?

[neon-ninja]

Yes. See #601 for information on the PytzUsageWarning. Yes.

[fernandonjardim]

Is there anywhere I can add "?sorting_setting=CHRONOLOGICAL" to the URL; so I can get all the posts by choronological order, instead of activity?