Messing Around with ChatGPT’s and Copilot’s Python Coding Abilities

Overview

I’ve been testing out ChatGPT’s capabilities for a few months with different tasks including quantitative and development questions. In a few cases, I’ll write the code and then see how ChatGPT solves it (or if it does).

This case was particularly interesting because I didn’t give it a bunch of instructions to start with and my request is a bit complicated because the solution entails OOP, class methods, filtering out the hidden methods, etc. And it solved it first try!

Then I asked it to expand the solution with two more methods, which it did no problem. And finally, I gave two general prompts to get ChatGPT to improve the solution. It cleaned up the redundant code and added docstrings, type hinting, etc. It also adds test cases in automatically without being asked AND it adds to those cases when I requested new features.

Overall, it impressed my here (and many other times), which is not a novel take!

ChatGPT vs Copilot for this use case

Just for fun, I gave the exact same prompts to Copilot. What I noticed in this use case is that both ChatGPT and Copilot produced workable solutions. ChatGPT was a better with technical details, and Copilot was better with semantics.

A couple of specific things I noticed:

  1. The results between ChatGPT and Copilot were both good, but ChatGPT was better overall (imo).
  2. ChatGPT’s initial method does not return the name of the method or the hidden dunder methods. Copilot’s solution does return the name of the util method (get_all_constants), which is not correct.
  3. Copilot did not do well with the prompt to get rid of redundant code. It changed the code in a trivial way, while ChatGPT understood exactly and made the correction; write a list comprehension for the dict method and then rely on that dict method via built in keys() and values() methods for dict to handle the other two class methods.
  4. Howeverrrrr, Copilot had better docstrings. For example,
    1. Copilot: """Get a dictionary with attribute names as keys and their values."""
    2. ChatGPT: """Returns a dictionary of all non-callable class attributes that do not start with '__'."""
  5. I also found Copilot’s naming to be more natural. For example, get_attributes (Copilot) vs get_all_constants (ChatGPT).

The chat with ChatGPT

(I’m including just the ChatGPT log to maintain readability and because it’s more technically impressive.)

Prompt 1

Prompt 1 (initial request): “I’m using a constants.py file to store constants in classes. For example,

class Constants:
A = "value"
B = "val"

I want to have a utility that will me a method in the class that returns a list of all the attributes. can you write that code?”

ChatGPT Answer 1: Certainly! You can create a utility method within the Constants class that returns a list of all class attributes. Here is how you can do it:

pythonCopy code
class Constants:
    A = "value"
    B = "val"

    @classmethod
    def get_all_constants(cls):
        return [attr for attr in dir(cls) if not callable(getattr(cls, attr)) and not attr.startswith("__")]

# Example usage:
constants_list = Constants.get_all_constants()
print(constants_list)  # Output: ['A', 'B']

In this code, the get_all_constants method iterates over all attributes of the class using dir(cls), filters out any callable attributes (methods) and any special attributes (those starting with __), and returns a list of the remaining attributes.

Prompt 2

Prompt 2 (more/new features): “cool, can you write a method to get all the attribute values and a method to get a dict with attributes and values?”

ChatGPT Answer 2: Sure! Here are the methods to get all the attribute values and a dictionary with attributes and values:

class Constants:
    A = "value"
    B = "val"

    @classmethod
    def get_all_constants(cls):
        return [attr for attr in dir(cls) if not callable(getattr(cls, attr)) and not attr.startswith("__")]

    @classmethod
    def get_all_values(cls):
        return [getattr(cls, attr) for attr in cls.get_all_constants()]

    @classmethod
    def get_constants_dict(cls):
        return {attr: getattr(cls, attr) for attr in cls.get_all_constants()}

# Example usage:
constants_list = Constants.get_all_constants()
print("Constants List:", constants_list)  # Output: ['A', 'B']

values_list = Constants.get_all_values()
print("Values List:", values_list)  # Output: ['value', 'val']

constants_dict = Constants.get_constants_dict()
print("Constants Dictionary:", constants_dict)  # Output: {'A': 'value', 'B': 'val'}

In this code:

  • get_all_values returns a list of all attribute values by using the get_all_constants method to get the attribute names and then fetching their values using getattr.
  • get_constants_dict returns a dictionary with attribute names as keys and their values as values by iterating over the attribute names from get_all_constants and creating a dictionary using a dictionary comprehension.

Prompt 3

Prompt 3 (request for improve + small hint): “Is there a way to make the methods simpler? It seems like they are repeating similar code between the methods”

ChatGPT Answer 3: Yes, we can simplify the methods by reducing repetition. Here’s a more concise version of the Constants class with simplified methods:

class Constants:
    A = "value"
    B = "val"

    @classmethod
    def get_constants(cls):
        return {attr: getattr(cls, attr) for attr in dir(cls) if not callable(getattr(cls, attr)) and not attr.startswith("__")}

    @classmethod
    def get_all_constants(cls):
        return list(cls.get_constants().keys())

    @classmethod
    def get_all_values(cls):
        return list(cls.get_constants().values())

# Example usage:
constants_list = Constants.get_all_constants()
print("Constants List:", constants_list)  # Output: ['A', 'B']

values_list = Constants.get_all_values()
print("Values List:", values_list)  # Output: ['value', 'val']

constants_dict = Constants.get_constants()
print("Constants Dictionary:", constants_dict)  # Output: {'A': 'value', 'B': 'val'}

In this version:

  • The get_constants method returns the dictionary with attribute names and their values.
  • The get_all_constants method now simply returns the keys of this dictionary.
  • The get_all_values method now returns the values of this dictionary.

This reduces code repetition and makes the methods more concise.

Prompt 4

Prompt 4 (asking for new feature/functionality): “nice. I want to have multiple classes that use these methods. What should I do?”

ChatGPT Answer 4: To allow multiple classes to use these methods, you can create a mixin class that provides these utility methods. Each class that needs these methods can then inherit from this mixin. Here’s how you can do it:

class ConstantsMixin:
    @classmethod
    def get_constants(cls):
        return {attr: getattr(cls, attr) for attr in dir(cls) if not callable(getattr(cls, attr)) and not attr.startswith("__")}

    @classmethod
    def get_all_constants(cls):
        return list(cls.get_constants().keys())

    @classmethod
    def get_all_values(cls):
        return list(cls.get_constants().values())

class ConstantsA(ConstantsMixin):
    A = "value"
    B = "val"

class ConstantsB(ConstantsMixin):
    X = "another_value"
    Y = "yet_another_value"

# Example usage:
constants_a_list = ConstantsA.get_all_constants()
print("ConstantsA List:", constants_a_list)  # Output: ['A', 'B']

values_a_list = ConstantsA.get_all_values()
print("ValuesA List:", values_a_list)  # Output: ['value', 'val']

constants_a_dict = ConstantsA.get_constants()
print("ConstantsA Dictionary:", constants_a_dict)  # Output: {'A': 'value', 'B': 'val'}

constants_b_list = ConstantsB.get_all_constants()
print("ConstantsB List:", constants_b_list)  # Output: ['X', 'Y']

values_b_list = ConstantsB.get_all_values()
print("ValuesB List:", values_b_list)  # Output: ['another_value', 'yet_another_value']

constants_b_dict = ConstantsB.get_constants()
print("ConstantsB Dictionary:", constants_b_dict)  # Output: {'X': 'another_value', 'Y': 'yet_another_value'}

In this code:

  • ConstantsMixin is a mixin class that provides the utility methods.
  • ConstantsA and ConstantsB are example classes that inherit from ConstantsMixin and define their own constants.
  • Each class can independently use the methods provided by ConstantsMixin to get its constants, values, and dictionary representation.

Prompt 5

Prompt 5 (general request to improve): “What do you think about this code? Would you improve or change anything?”

ChatGPT Answer 5: The code you have is functional and well-organized. However, there are a few improvements and enhancements that could make it more robust and clear:

  1. Documentation: Adding docstrings to the mixin methods can improve code readability and maintenance.
  2. Type Annotations: Including type annotations for better code clarity and to help with static analysis tools.
  3. Edge Case Handling: Ensure that the methods handle edge cases, such as when there are no constants defined.

Here is the revised code with these enhancements:

class ConstantsMixin:
    @classmethod
    def get_constants(cls) -> dict:
        """
        Returns a dictionary of all non-callable class attributes that do not start with '__'.
        """
        return {attr: getattr(cls, attr) for attr in dir(cls) if not callable(getattr(cls, attr)) and not attr.startswith("__")}

    @classmethod
    def get_all_constants(cls) -> list:
        """
        Returns a list of all non-callable class attribute names that do not start with '__'.
        """
        return list(cls.get_constants().keys())

    @classmethod
    def get_all_values(cls) -> list:
        """
        Returns a list of all values of non-callable class attributes that do not start with '__'.
        """
        return list(cls.get_constants().values())

class ConstantsA(ConstantsMixin):
    """
    Example class to demonstrate usage of ConstantsMixin.
    """
    A = "value"
    B = "val"

class ConstantsB(ConstantsMixin):
    """
    Another example class to demonstrate usage of ConstantsMixin.
    """
    X = "another_value"
    Y = "yet_another_value"

# Example usage:
constants_a_list = ConstantsA.get_all_constants()
print("ConstantsA List:", constants_a_list)  # Output: ['A', 'B']

values_a_list = ConstantsA.get_all_values()
print("ValuesA List:", values_a_list)  # Output: ['value', 'val']

constants_a_dict = ConstantsA.get_constants()
print("ConstantsA Dictionary:", constants_a_dict)  # Output: {'A': 'value', 'B': 'val'}

constants_b_list = ConstantsB.get_all_constants()
print("ConstantsB List:", constants_b_list)  # Output: ['X', 'Y']

values_b_list = ConstantsB.get_all_values()
print("ValuesB List:", values_b_list)  # Output: ['another_value', 'yet_another_value']

constants_b_dict = ConstantsB.get_constants()
print("ConstantsB Dictionary:", constants_b_dict)  # Output: {'X': 'another_value', 'Y': 'yet_another_value'}

Explanation of Improvements:

  1. Docstrings: Each method now has a docstring that describes what the method does, which improves readability and helps other developers understand the purpose of each method.
  2. Type Annotations: Methods now include return type annotations to specify what type of data they return. This can be beneficial for readability and for static analysis tools.
  3. Edge Case Handling: The current implementation already handles edge cases well by filtering out non-callable attributes and those starting with __. Additional handling would depend on specific needs or edge cases relevant to your application.

These changes make the code clearer and more maintainable, especially when used in larger projects or by teams.