The getBack() function is created to deal with the unsolved mystery of clicking into an unwanted page.Pain points with collecting Facebook data include: 1) login required from the very beginning - no way around it 2) many buttons to click in order to get sufficient (and usable) data, mainly of 2 types - “See More” and expand comments/replies (there are also “See More”’s in comments/replies) 3) it is not only hard to distinguish these buttons we want to click from the ones we don’t (FB’s HTML class naming is far from intuitive to begin with), but it is also very easy to accidentally click on an unwanted button that takes you entirely off track (and in my case, that means having to start over - I haven’t figured out an elegant way to solve this) I’m leaving it here for any visitors or my future self to improve! The following process is what made sense to me but may not be the most elegant or efficient. Sites are involving (FB especially) and better, smarter ways are always available. It’s certainly a work in progress, as always in the case of web scraping.
#Facebook group data extractor code#
These review data could be extremely helpful to competitor or benchmark analysis insights could be generated from conducting text analysis or examining the interaction among the commenters, each of whom has an accessible social profile and varied social influence - another layer of analysis enabled by social networks.Īfter spending quite some time dissecting Facebook page structure and trying to figure out dozens of workarounds, this post is to serve as a summary of the process for myself and a showcase of the code (as of now) for anyone who might want to customize and build their own scrapers.
Readily available tools, such as this FB page scraper, are useful in getting the standard posts and basic metadata, but limited in other use cases, such as the one I have at hand - getting reviews from a public page together with all the comments and replies, such as this Universal Studios Hollywood page. Unsatisfied with closed doors, I set out again to automatically get data from public Facebook pages. Basically, anything that you are not an owner of is impossible to get without an app review, which makes life difficult to those needing social media data for academic research purposes as the development of an app is often not attainable or relevant, let alone the more convoluted review process.
Collecting Public Data from Facebook Using Selenium and Beautiful Soupĭespite having an API, it is getting increasingly difficult to get data from Facebook - even the most transparent, public, basic information.
0 kommentar(er)
