Restrictions on very large texts

The OpenAI API (currently) limits the maximum number of tokens per request depending on the model used (currently the maximum possible tokens are 32,768 by using the gpt-4-32k model). You can find a detailed overview of models and the maximum number of tokens here:

https://platform.openai.com/docs/models/

Based on the current status of the extension, it is unfortunately not yet possible to analyze larger texts.

Possible limitations when using the GPT-4 model

At the time of version 0.6.0 release, only people with a $1 or more successful payment on their account have access to the GPT-4 model. According to OpenAI, that should change soon.

If you try to use the GPT-4 model without access you currently get a 404 error message which is a bit confusing.

Add via composer.json:

composer require "passionweb/ai-seo-helper"

• Install the extension via composer
• Flush TYPO3 and PHP Cache
• Add your OpenAI secret key to the extension configuration before using the extension

Add via TER:

If you want to install the extension via TER you can find detailed instructions here.

• Install the extension via TER
• Flush TYPO3 and PHP Cache
• Add your OpenAI secret key to the extension configuration before using the extension

Further information

The different ways to install an extension and additional detailed information can be found here.

Extension manager configuration

You can adapt the following parameters to your personal needs. After the first tests, the best results were achieved with the predefined values. However, this is no guarantee that these values ​​will also achieve the best results for you.

# cat=API Key; type=string; label=OpenAI Secret Key
openAiApiKey = YOUR_API_KEY

Enter your generated API key.

# cat=meta description; type=string; label=Prompt-Prefix for meta description generation
openAiPromptPrefixMetaDescription = Extract five seo meta descriptions in a bullet point list, each seo meta description in one short sentence and with a maximum of 150 characters or less, for the content of

Enter your instruction for generating meta description. Since OpenAI calculates the length of the content with tokens (an explanation of the conversion of tokens into characters and sentences can be found here) by default, we have to explicitly tell the AI ​​the desired total length and the type of expected creation

# cat=keywords; type=string; label=Prompt-Prefix for keywords generation
openAiPromptPrefixKeywords = Extract seo keywords from this text

Enter your instruction for generating keywords.

# cat=keywords; type=string; label=Replace first part of generated keywords
replaceTextKeywords = SEO keywords:

The content generated by OpenAI is usually supplemented with a short introduction. Here you can define the part of the generated content that should be removed.

#cat=page title; type=string; label=Prompt-Prefix for page title suggestions generation
openAiPromptPrefixPageTitle = Suggest page title ideas in bullet point list for this text

Enter your instruction for generating page title suggestions (IMPORTANT: response must be a bullet point list as the return is processed that way).

#cat=open graph; type=string; label=Prompt-Prefix for Open Graph title suggestions generation
openAiPromptPrefixOgTitle = Suggest Open Graph title ideas in bullet point list for this text

Enter your instruction for generating Open Graph title suggestions (IMPORTANT: response must be a bullet point list as the return is processed that way).

# cat=open graph; type=string; label=Prompt-Prefix for Open Graph description suggestions generation
openAiPromptPrefixOgDescription = Extract five Open Graph descriptions in a bullet point list, each Open Graph description in one short sentence and with a maximum of 150 characters or less, for the content of

Enter your instruction for generating Open Graph description suggestions. Since OpenAI calculates the length of the content with tokens (an explanation of the conversion of tokens into characters and sentences can be found here) by default, we have to explicitly tell the AI the desired total length and the type of expected creation

#cat=twitter; type=string; label=Prompt-Prefix for Twitter title suggestions generation
openAiPromptPrefixTwitterTitle = Suggest Twitter title ideas in bullet point list for this text

Enter your instruction for generating Twitter title suggestions (IMPORTANT: response must be a bullet point list as the return is processed that way).

# cat=twitter; type=string; label=Prompt-Prefix for Twitter description suggestions generation
openAiPromptPrefixTwitterDescription = Extract five Twitter descriptions in a bullet point list, each Twitter description in one short sentence and with a maximum of 150 characters or less, for the content of

Enter your instruction for generating Twitter description suggestions. Since OpenAI calculates the length of the content with tokens (an explanation of the conversion of tokens into characters and sentences can be found here) by default, we have to explicitly tell the AI the desired total length and the type of expected creation

By default, the extension prepares the Twitter description suggestions in such a way that they can be selected via radio button. If you change the prompt prefix and no bullet point list is returned as a result, display problems can occur here. With this option you can output the raw content and select your favorite Twitter description via copy/paste.

#cat=open graph & twitter; type=boolean; label=Fill Open Graph and Twitter titles
pageTitleForOgAndTwitter = 0

Use selected page title suggestion for Open Graph and Twitter titles

#cat=open graph & twitter; type=boolean; label=Fill Open Graph and Twitter descriptions
metaDescriptionForOgAndTwitter = 0

Use selected meta description suggestion for Open Graph and Twitter descriptions

# cat=basic request settings; type=string; label=OpenAI Model
openAiModel = gpt-3.5-turbo

The id of the model which will generate the completion. See models overview for an overview of available models.

# cat=basic request settings; type=double+; label=OpenAI Temperature
openAiTemperature = 0.5

What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.

# cat=basic request settings; type=int+; label=OpenAI Max-Tokens
openAiMaxTokens = 275

The token (what are tokens and how to count them) count of your prompt plus max_tokens cannot exceed the model's context length. Most models have a context length of 2048 tokens (except for the newest models, which support 4096).

# cat=basic request settings; type=int+; label=OpenAI Top-P
openAiTopP = 1

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

# cat=basic request settings; type=double; label=OpenAI Frequency Penalty
openAiFrequencyPenalty = 0.8

Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

# cat=basic request settings; type=double; label=OpenAI Presence Penalty
openAiPresencePenalty = 0

Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

# cat=basic request settings; type=boolean; label=Use always URL for requests
useUrlForRequest = 1

With this option you can use the corresponding URL of the page for all analyses. As a result, you have to use fewer tokens to carry out your corresponding analyses. IMPORTANT: The page must be publicly accessible (hidden pages fail and pages in a local environment lead to poor results)

# cat=news; type=string; label=Enter the ID of one page that is used as a detail view for news articles of the news extension
singleNewsDisplayPage =

Enter the ID of one page that is used as a detail view for news articles of EXT:news. This page will be used to generate the BackendPreviewUrl.

# cat=news; type=string; label=Prompt-Prefix for meta description suggestions generation of news article
openAiPromptPrefixNewsMetaDescription = Extract five seo meta descriptions in a bullet point list, each seo meta description in one short sentence and with a maximum of 150 characters or less, for the content of

Enter your instruction for generating meta description suggestions. Since OpenAI calculates the length of the content with tokens (an explanation of the conversion of tokens into characters and sentences can be found here) by default, we have to explicitly tell the AI the desired total length and the type of expected creation

# cat=news; type=string; label=Prompt-Prefix for keywords generation of news article
openAiPromptPrefixKeywords = Extract seo keywords from this news article. Return the result in a comma separated list.

Enter your instruction for generating keywords.

# cat=news; type=string; label=Replace first part of generated keywords of news article
replaceTextNewsKeywords = SEO keywords:

The content generated by OpenAI is usually supplemented with a short introduction. Here you can define the part of the generated content that should be removed.

# cat=news; type=string; label=Prompt-Prefix for alternative title suggestions generation
openAiPromptPrefixNewsAlternativeTitle = Suggest page title ideas in bullet point list for this text

Enter your instruction for generating alternative title suggestions (IMPORTANT: response must be a bullet point list as the return is processed that way).

Add TCA configuration to Configuration/TCA/Overrides/pages.php

$GLOBALS['TCA']['pages']['columns']['FIELD_IDENTIFIER']['config'] = array_merge_recursive(
    $GLOBALS['TCA']['pages']['columns']['FIELD_IDENTIFIER']['config'],
    [
        'fieldControl' => [
            'importControl' => [
                'renderType' => 'NODE_IDENTIFIER'
            ]
        ]
    ]
);

Add FormEngine fieldControl (e.g. to Classes/FormEngine/FieldControl/AiSeoFIELD_IDENTIFIER_UPPER_CAMELCASE.php)

class AiSeoFIELD_IDENTIFIER_UPPER_CAMELCASE extends AbstractNode
{
    public function render(): array
    {
        $resultArray = [
            'iconIdentifier' => 'actions-document-synchronize',
            'title' => 'Your custom title',
            'linkAttributes' => [
                'id' => 'FIELD_IDENTIFIER_generation',
                'class' => 'ai-seo-helper-suggestions-generation-btn',
                'data-page-id' => $this->data['databaseRow']['uid'],
                'data-field-name' => 'FIELD_IDENTIFIER'
            ]
        ];

        $javaScriptModuleService = GeneralUtility::makeInstance(JavaScriptModuleService::class);

        return array_merge($resultArray, $javaScriptModuleService->addModules());
    }
}

Add registration in NodeFactory to ext_localconf.php

$GLOBALS['TYPO3_CONF_VARS']['SYS']['formEngine']['nodeRegistry'][TIMESTAMP] = [
    'nodeName' => 'NODE_IDENTIFIER',
    'priority' => 30,
    'class' => \Vendor\Package\FormEngine\FieldControl\AiSeoFIELD_IDENTIFIER_UPPER_CAMELCASE::class
];

Add further configuration settings to ext_conf_template.txt

#cat=custom category; type=string; label=Your custom title
openAiPromptPrefixFIELD_IDENTIFIER_UPPER_CAMELCASE = Your custom prompt

Add function to controller (e.g. to Classes/Controller/Ajax/AiController.php)

public function generateFIELD_IDENTIFIER_UPPER_CAMELCASEAction(ServerRequestInterface $request): ResponseInterface
{
    return $this->generateSuggestions($request, 'FIELD_IDENTIFIER_UPPER_CAMELCASE');
}

The function generateSuggestions() can be used from \Passionweb\AiSeoHelper\Service\ContentService

Add ajax route to Configuration/Backend/AjaxRoutes.php

return [
    'FIELD_IDENTIFIER_generation' => [
        'path' => 'CUSTOM_PATH',
        'target' => \Vendor\Package\Controller\Ajax\AiController::class . '::generateFIELD_IDENTIFIER_UPPER_CAMELCASEAction'
    ],
];

Once all adjustments have been made, flush the TYPO3 and PHP cache and test if everything works as desired.

Generate meta description (suggestions)

Added an additional button next to the meta description text field. When you click this button, the (text) content of the selected page is generated, and you get meta description suggestions with the help of the AI. By default, the extension prepares the meta description suggestions in such a way that they can be selected via radio button. If you change the prompt prefix and no bullet point list is returned as a result, display problems can occur here. If you set the option showRawMetaDescriptionSuggestions to true within the extension configuration you can output the raw content and select your favorite meta description via copy/paste.

It can happen that the AI returns texts that exceed the maximum allowed length of the meta description. To additionally check the length of the meta description, the extension Yoast SEO for TYPO3 can be used, for example, or various online tools.

Generate keywords

Added an additional button next to the keywords text field. When you click this button, the (text) content of the selected page is generated and keywords that is as suitable as possible is created with the help of the AI. Currently, the page must not be deactivated in the backend. Depending on the page size, the process may take a few seconds. However, notifications are used to display appropriate information.

Generate page title (suggestions)

Added an additional button next to the seo title text field. When you click this button, the (text) content of the selected page is generated and you get page title suggestions with the help of the AI. By default, the extension prepares the page title suggestions in such a way that they can be selected via radio button. If you change the prompt prefix and no bullet point list is returned as a result, display problems can occur here. If you set the option showRawPageTitleSuggestions to true within the extension configuration you can output the raw content and select your favorite page title via copy/paste.

Generate Open Graph title (suggestions)

Added an additional button next to the Open Graph title text field. When you click this button, the (text) content of the selected page is generated, and you get Open Graph title suggestions with the help of the AI. By default, the extension prepares the Open Graph title suggestions in such a way that they can be selected via radio button. If you change the prompt prefix and no bullet point list is returned as a result, display problems can occur here. If you set the option showRawOgTitleSuggestions to true within the extension configuration you can output the raw content and select your favorite Open Graph title via copy/paste.

Generate Open Graph description (suggestions)

Added an additional button next to the Open Graph description text field. When you click this button, the (text) content of the selected page is generated, and you get Open Graph description suggestions with the help of the AI. By default, the extension prepares the Open Graph description suggestions in such a way that they can be selected via radio button. If you change the prompt prefix and no bullet point list is returned as a result, display problems can occur here. If you set the option showRawOgDescriptionSuggestions to true within the extension configuration you can output the raw content and select your favorite Open Graph description via copy/paste.

Generate Twitter title (suggestions)

Added an additional button next to the Twitter title text field. When you click this button, the (text) content of the selected page is generated, and you get Twitter title suggestions with the help of the AI. By default, the extension prepares the Twitter title suggestions in such a way that they can be selected via radio button. If you change the prompt prefix and no bullet point list is returned as a result, display problems can occur here. If you set the option showRawTwitterTitleSuggestions to true within the extension configuration you can output the raw content and select your favorite Twitter title via copy/paste.

Generate Twitter description (suggestions)

Added an additional button next to the Twitter description text field. When you click this button, the (text) content of the selected page is generated, and you get Twitter description suggestions with the help of the AI. By default, the extension prepares the Twitter description suggestions in such a way that they can be selected via radio button. If you change the prompt prefix and no bullet point list is returned as a result, display problems can occur here. If you set the option showRawTwitterDescriptionSuggestions to true within the extension configuration you can output the raw content and select your favorite Twitter description via copy/paste.

Use page title suggestion for Open Graph and Twitter titles

Since Version 0.5.0 you have the option to copy the selected page title suggestion to the fields for Open Graph and Twitter titles (can be found within the tab "Social media"). Therefore you must enable the option pageTitleForOgAndTwitter in the extension settings. If you select a page title the content will be copied to the fields og_titleand twitter_title too.

Use meta description suggestion for Open Graph and Twitter descriptions

Since Version 0.5.0 you have the option to copy the selected meta description suggestion to the fields for Open Graph and Twitter descriptions (can be found within the tab "Social media"). Therefore you must enable the option metaDescriptionForOgAndTwitter in the extension settings. If you select a meta description the content will be copied to the fields og_descriptionand twitter_description too.

Generate meta description (suggestions)

Added an additional button next to the meta description text field. When you click this button, the (text) content of the selected news article is generated, and you get meta description suggestions with the help of the AI. By default, the extension prepares the meta description suggestions in such a way that they can be selected via radio button. If you change the prompt prefix and no bullet point list is returned as a result, display problems can occur here. If you set the option showRawNewsMetaDescriptionSuggestions to true within the extension configuration you can output the raw content and select your favorite meta description via copy/paste.

It can happen that the AI returns texts that exceed the maximum allowed length of the meta description. To additionally check the length of the meta description, the extension ["Yoast SEO for TYPO3"](https://extensions.typo3.org/extension/yoast_seo "Yoast SEO for TYPO3") can be used, for example, or various online tools.

Generate keywords

Added an additional button next to the keywords text field. When you click this button, the (text) content of the selected news article is generated and keywords that is as suitable as possible is created with the help of the AI. Currently, the page must not be deactivated in the backend. Depending on the page size, the process may take a few seconds. However, notifications are used to display appropriate information.

Generate alternative title (suggestions)

Added an additional button next to the alternative title text field. When you click this button, the (text) content of the selected news article is generated, and you get alternative title suggestions with the help of the AI. By default, the extension prepares the alternative title suggestions in such a way that they can be selected via radio button. If you change the prompt prefix and no bullet point list is returned as a result, display problems can occur here. If you set the option showRawNewsAlternativeTitleSuggestions to true within the extension configuration you can output the raw content and select your favorite page title via copy/paste.

Analyzing the page content based on the text

As already mentioned, this option is primarily limited by the length of the allowed characters per OpenAI request. Furthermore, using this method requires the use of a comparatively large number of tokens.

The following settings are necessary:

• disable the useUrlForRequest option in the extension settings
• check if the ISO code(s) of the language configurations is/are present in the predefined fields (if not present, custom languages can be added, see Add custom languages).
• Definition of the corresponding prompt in English.

The entire prompt is then assembled from the prompt prefix (from the extension settings), the page url and the language suffix (based on the language used on the page). An example of the generated prompt for page title suggestions would look like this (for a German language site):

1. Suggest page title ideas in bullet point list for the following text (content from extension setting openAiPromptPrefixPageTitle)
2. in German (language suffix based on the language used on the page)
3. Here comes the page content

Complete:

Suggest page title ideas in bullet point list for the following text in German:

Here comes the page content

Analyzing the page content by URL

In contrast to the text-based variant, this option uses the URL of the page to analyze the desired data. If you want to use this variant, the following settings are necessary:

• enable the useUrlForRequest option in the extension settings
• check if the ISO code(s) of the language configurations is/are present in the predefined fields (if not present, custom languages can be added, see Add custom languages).
• Definition of the corresponding prompt in English.

The entire prompt is then assembled from the prompt prefix (from the extension settings), the page url and the language suffix (based on the language used on the page). An example of the generated prompt for page title suggestions would look like this (for a German language site):

1. Suggest page title ideas in bullet point list for (content from extension setting openAiPromptPrefixPageTitle)
2. https://www.example.de/ (page url)
3. in German (language suffix based on the language used on the page)

Complete:

Suggest page title ideas in bullet point list for https://www.example.de/ in German

Requirement is that the page is publicly accessible (hidden pages fail and pages in a local environment lead to poor results).

A major advantage is that this variant saves quite a lot of OpenAI tokens (and thus costs), since only the URL is sent to OpenAI instead of the entire page content