The Question is no longer "How Do I Build X?"
The question is now "what shall (and what should) we build?"
The following is a transcript of a recent prompting session I had using GPT-4. The transcript is unedited except for formatting to make the code blocks display correctly.
SYSTEM
You are a programming assistant. You write code in response to user specifications.
USER
I want to write a Python program to scrape a specific webpage each time it is run and return data from three specific id elements on the page.
The website is: https://lectionary.anglican.ca
The page segment i want to get the text data from are the following:
id="lectionary_MP"
id="lectionary_HE"
id="lectionary_EP"
Please write me a Python script to accomplish this.
ASSISTANT
To accomplish this, you can use a Python library called Beautiful Soup for web scraping and the requests library to get the webpage content. First, you need to install both libraries if you haven't already:
```bash
pip install beautifulsoup4
pip install requests
```
Here's the Python script to scrape the specified webpage and extract the text data from the given elements:
import requests
from bs4 import BeautifulSoup
def get_lectionary_data():
url = 'https://lectionary.anglican.ca'
response = requests.get(url)
if response.status_code == 200:
soup = BeautifulSoup(response.content, 'html.parser')
lectionary_MP = soup.find(id='lectionary_MP').get_text(strip=True)
lectionary_HE = soup.find(id='lectionary_HE').get_text(strip=True)
lectionary_EP = soup.find(id='lectionary_EP').get_text(strip=True)
print("Lectionary MP:", lectionary_MP)
print("Lectionary HE:", lectionary_HE)
print("Lectionary EP:", lectionary_EP)
else:
print("Error: Unable to get webpage content")
if __name__ == '__main__':
get_lectionary_data()
This script defines a function `get_lectionary_data()` that fetches the webpage content, and using Beautiful Soup, it finds the elements with the specified ids and extracts their text content. The script then prints the data for each of the elements.
USER
Now we're going to write a second script that will use the output of the first one as input. This script will take text like the following:
Lectionary MP: Morning Prayer:Ps 95 (Invitatory), 102; Two of the following three readings: Jer 23:1-8; Rom 8:28-39; Jn 6:52-59; Coll 291
Lectionary HE: Holy Eucharist:As Sunday, except: Wis 2:1a, 12-22; Ps 34:15-22; Jn 7:1-2, 10, 25-30
Lectionary EP: Evening Prayer:Eve of the Annunciation: Ps 8, 138; Gen 3:1-15; Rom 5:12-21 or Gal 4:1-7; Coll 403 or FAS 125
From the text, the script will extract only the bible verses (those formatted as Jer 23:1-8, Rom 8:28-39, and so forth) and print them for the user to see.
Assistant: Here's a Python script that takes the output from the previous script as input and extracts the Bible verses using regular expressions:
import re
def extract_bible_verses(text):
pattern = r'\b(?:[1-3]\s)?[A-Za-z]+ \d{1,3}(?::\d{1,3}(?:-\d{1,3})?)?\b'
bible_verses = re.findall(pattern, text)
return bible_verses
def main():
input_text = '''Lectionary MP: Morning Prayer:Ps 95 (Invitatory), 102; Two of the following three readings: Jer 23:1-8; Rom 8:28-39; Jn 6:52-59; Coll 291
Lectionary HE: Holy Eucharist:As Sunday, except: Wis 2:1a, 12-22; Ps 34:15-22; Jn 7:1-2, 10, 25-30
Lectionary EP: Evening Prayer:Eve of the Annunciation: Ps 8, 138; Gen 3:1-15; Rom 5:12-21 or Gal 4:1-7; Coll 403 or FAS 125'''
extracted_verses = extract_bible_verses(input_text)
print("Bible Verses:")
for verse in extracted_verses:
print(verse)
if __name__ == '__main__':
main()
This script defines a function `extract_bible_verses()` that takes the input text and uses a regular expression pattern to find all occurrences of Bible verse references. The `main()` function demonstrates how to use this function with a sample input text. You can replace the `input_text` variable with the actual output from the first script.
User
One more script. This one will go to https://www.biblegateway.com and extract the bible verses from the page.
As an example, the URL to get the verse Jer 23:1-8 in the New Revised Standard Edition, Anglicised translation is: :https://www.biblegateway.com/passage/?search=Jer+23%3A1-8&version=NRSVA"
Please write a script to take a string input such as "Jer 23:1-8" or "Ps 95", go to Bible Gateway, and scrape the text for the passage. As a guide, the class element on the page that contains the Bible text is: class="version-NRSVA result-text-style-normal text-html".
Assistant
Here's a Python script to scrape the Bible passage text from Bible Gateway based on a given reference:
import requests
from bs4 import BeautifulSoup
def get_bible_passage(reference, version='NRSVA'):
base_url = 'https://www.biblegateway.com/passage/'
params = {
'search': reference,
'version': version
}
response = requests.get(base_url, params=params)
if response.status_code == 200:
soup = BeautifulSoup(response.content, 'html.parser')
bible_text_element = soup.find(class_='version-{} result-text-style-normal text-html'.format(version))
if bible_text_element is None:
print("Error: Unable to find Bible passage.")
return
bible_text = bible_text_element.get_text(separator='\n')
return bible_text
else:
print("Error: Unable to get webpage content")
def main():
reference = 'Jer 23:1-8'
passage_text = get_bible_passage(reference)
if passage_text:
print("Bible Passage ({}):".format(reference))
print(passage_text)
if __name__ == '__main__':
main()
This script defines a function `get_bible_passage()` that takes a Bible reference and an optional version (default is 'NRSVA'). It constructs the URL for Bible Gateway and fetches the webpage content. Then, using Beautiful Soup, it finds the element with the specified class and extracts the text content. The `main()` function demonstrates how to use this function with a sample reference. You can replace the `reference` variable with any valid Bible verse reference.
Reflections
I turned the above into a complete, end to end script, available on my Github page: https://github.com/HamiltonianGraph/dailyoffice. I did not ask GPT-4 to make an end to end solution out of the above - I did the combining and feeding of each output into the subsequent input myself. Partly I did this because I had a strong idea how I wanted the code to read. Next time I'll probably just ask GPT-4 to bring the pieces together.
For my non-religious readers, this will seem an uninteresting application. For my religious readers, you’re probably asking “why don’t you just flip through your Bible like the rest of us do?”
To answer the first: the important point is that I conceptualized this over breakfast, jotted a few quick notes, fired up GPT-4, and had a working program about 20 minutes after the conversation began. I could have coded this myself - it’s not using any principles or modules I’m not familiar with (I have, to quote some friends, a ‘sick’ interest in coding regular expressions). What else could I, could you, could everyone, build?
For the religious readers, I do read my printed Bible, but some mornings the effort of looking up the passages is a deterrent to doing so. Writing the above gives me a way to do my readings more frequently, which I find to be better for my spiritual life.
My new creative challenge is, while I wait for access to the ChatGPT plugins, is coming up with new things to ask GPT-4 to program. Normally, the thought of a new program to build is followed by the “and how much time/effort will take?” Now, for every application I have been able to think of so far, GPT-4 provides a rapid means to prototype it. Even when I don’t use the output, I still get a customized coding sample that gives me ideas for how to approach the problem I want to solve.
My advice is, whether you’re working in a technical field or not, start experimenting now. Start imaging how these tools can assist you in your job.