Version 12.71 update #3748

Zbizu · 2021-10-18T17:54:48Z

I have followed proper The Forgotten Server code styling.
I have read and understood the contribution guidelines before making this PR.
I am aware that this PR may be closed if the above-mentioned criteria are not fulfilled.

Description

~~Work in progress. It's working but the code is nowhere near its final form. Treat it as a sketch, a proof of concept for now.~~
~~Visit #3747 to take part of important decision regarding this change.~~

The code is ready for testing! Details can be found in comments below.

Changes Proposed

12.71 support
~~10.98 backwards compatibility~~ This feature was voted out.
cid being dependent on GUID (this is necessary to save containers)
adjust monster/npc generated ids (the cid)
standardized creature id bounds (uses variable instead of hardcoded number now)
improved code behind mounts
updated summon status refresh packet
updated talkactions message types (blue console message is deprecated)
feature: packet sequencing
feature: login with email (you can set @admin as your email btw or if you're running otclient, still use your account name)
feature: right click to talk with npc
feature: save opened containers
feature: mount colors
feature: session end packet
feature: cid and GUID in onLook (gm/god only)
new items
new outfits
new mounts
new NPC bubbles
new condition icons (enums only for now)
new message types

Issues addressed: #3492, followup to #3671

Credits

Special thanks to:

OTBR community (especially people involved in "Global" and "Canary" projects and their OTClient fork) for providing the code and tools necessary to achieve that
SaiyansKing for paving the way for QT clients
Arch-Mina for providing version 12 tools to work with client
EPuncker - providing spr/dat, help with otb, game content information, testing and help with login.php
kokokoko - updating items.otb with new items and data
omarcopires - review, testing and help with login.php
TFS maintainers for ensuring code quality and helping with project direction

logging in now possible both ways - legacy and webService\nClient 12 will crash on login because the server still sends 10.98 packets.

Not necessary unless you run multiworld loginserver. Not present in current tfs either.

updated otb to version 12. You need special spr/dat to play with 10.98 now. Client 12 will still crash in most situations.

Zbizu · 2021-10-19T16:08:03Z

Version 12 is ready for testing.
10.98 compatibility dropped.

To log in, use Znote AAC with this change applied:
https://github.com/Znote/ZnoteAAC/pull/493/files (Updated)
(the packet has to look like this: login\npassword\ntoken or empty\ntimestamp divided by 30)

Please post bugs if you find any.

Known bugs:

~~sell price not displaying properly in NPC trade~~
~~hidden creatures still display name~~
~~displaying podium in game screen is very likely to crash the client due to lack of proper bytes in the packet~~
~~browse field crashes the client and I have no idea why~~
~~battle list is always on and there are weird missing containers after login~~

Known missing features:

everything added after 10.98

Things to adjust:

[protocolgame] ~~use podium ItemId/flag once we get proper otb~~ Update: podium will be added as a followup to this pr.
(optional/cosmetic) ~~[protocolgame] use purse ItemId once we get proper otb~~ to do: Update purse Id const

mattyx14 · 2021-10-19T16:29:12Z

I would just like to say that you try to make both items.otb the same 🙏🏼

beats-dh · 2021-10-26T12:40:37Z

It looks like a great job!

Zbizu · 2021-10-26T12:41:21Z

action bars and saved containers fully functional now

src/const.h

src/iologindata.cpp

src/item.cpp

src/server.cpp

src/protocol.cpp

nekiro · 2021-11-03T15:39:00Z

src/protocolgame.cpp

@@ -356,16 +372,31 @@ void ProtocolGame::onRecvFirstMessage(NetworkMessage& msg)
 		writeToOutputBuffer(opcodeMessage);
 	}

-	msg.skipBytes(1); // gamemaster flag
+	// Change packet verifying mode for QT clients
+	if (version >= 1111 && operatingSystem >= CLIENTOS_QT_LINUX && operatingSystem <= CLIENTOS_QT_MAC) {


we aren't allowing <= 1111, so I guess this check is redundant and checksum mode should be sequence by default, which would also delete enum for CHECKSUM_SEQUENCE

otclient compatible with protocol 1271 still uses adler32
version check is required to send "only x protocol allowed" message properly (it needs a proper header)
qt client was using checksums early too

wasn't otclient supposed to be the base client for tfs by the way?

Well then I would say otc should be fixed to act like qt client.

Well then I would say otc should be fixed to act like qt client.

Is there development on their behalf? OTC shouldn't block TFS but it would be nice to have them move together.

src/protocolgame.cpp

src/itemloader.h

ghost

I've been testing it for days already, everything fine so far

Zbizu · 2021-11-06T15:27:38Z

context for most recent commit: the "risky" symbol (the spark under guild emblem) is no longer used, the "helpers" packet got repurposed to show locker search results, which resulted in client crash when inviting to party. This commit fixed that.

Znote · 2021-11-07T15:57:14Z

src/iologindata.cpp

@@ -77,7 +77,7 @@ bool IOLoginData::loginserverAuthentication(const std::string& name, const std::
 {
 	Database& db = Database::getInstance();

-	DBResult_ptr result = db.storeQuery(fmt::format("SELECT `id`, `name`, `password`, `secret`, `type`, `premium_ends_at` FROM `accounts` WHERE `name` = {:s}", db.escapeString(name)));
+	DBResult_ptr result = db.storeQuery(fmt::format("SELECT `id`, `name`, `password`, `secret`, `type`, `premium_ends_at` FROM `accounts` WHERE `name` = {:s} OR `email` = {:s}", db.escapeString(name), db.escapeString(name)));


Is this faster than:

Suggested change

DBResult_ptr result = db.storeQuery(fmt::format("SELECT `id`, `name`, `password`, `secret`, `type`, `premium_ends_at` FROM `accounts` WHERE `name` = {:s} OR `email` = {:s}", db.escapeString(name), db.escapeString(name)));

DBResult_ptr result = db.storeQuery(fmt::format("SELECT `id`, `name`, `password`, `secret`, `type`, `premium_ends_at` FROM `accounts` WHERE {:s} IN(`name`, `email`)", db.escapeString(name)));

IN statements saves us an db.escapestring, but perhaps it breaks index scanning?
Perhaps we should make an index that stores both name and email.

Just store the result of db.escapeString if you don't want to call it twice

Znote · 2021-11-07T15:58:36Z

src/iologindata.cpp

@@ -104,8 +104,7 @@ bool IOLoginData::loginserverAuthentication(const std::string& name, const std::
 uint32_t IOLoginData::gameworldAuthentication(const std::string& accountName, const std::string& password, std::string& characterName, std::string& token, uint32_t tokenTime)
 {
 	Database& db = Database::getInstance();
-
-	DBResult_ptr result = db.storeQuery(fmt::format("SELECT `id`, `password`, `secret` FROM `accounts` WHERE `name` = {:s}", db.escapeString(accountName)));
+	DBResult_ptr result = db.storeQuery(fmt::format("SELECT `id`, `password`, `secret` FROM `accounts` WHERE `name` = {:s} OR `email` = {:s}", db.escapeString(accountName), db.escapeString(accountName)));


See suggestion iologindata.cpp#80

Znote · 2021-11-07T16:07:59Z

src/creature.h

@@ -71,6 +71,8 @@ class Tile;
 static constexpr int32_t EVENT_CREATURECOUNT = 10;
 static constexpr int32_t EVENT_CREATURE_THINK_INTERVAL = 1000;
 static constexpr int32_t EVENT_CHECK_CREATURE_INTERVAL = (EVENT_CREATURE_THINK_INTERVAL / EVENT_CREATURECOUNT);
+static constexpr uint32_t CREATURE_ID_MIN = 0x10000000;


What does this have to do with protocol 12? Should be a separate PR

it actually does and I think he explained why, there were some changes regarding ids in new client I believe

Znote

There is 74 commits in this commit history, 17+? of which are cleanups.
Since this is a big PR, we want to do a proper merge (this will give you credit for each commit instead of just one squashed one).

But we need each commit to be purpose specific and well explained, some commits could have a better description, some commits could be squashed together. (etc, 1 cleanup commit if any instead of 17, with a deeper description of what that involves).

A rebase against otland/master will remake these commits to follow after the latest commit in master branch, by using git rebase -i master (after the local master branch is up to date with otland/master) you can resort the order of your commits, and by changing pick #hash to squash #hash will squash a commit to its previous commit.

I suggest you play around with this in a version12_c branch, so this becomes a backup branch while your learning the ropes.

Normal rebase to get your commits after master: https://www.youtube.com/watch?v=f1wnYdLEpgI

In your branch, you have synced your branch to master, these commits are cluttered between your own feature branch commits, for instance the commits at october 23: https://github.com/Zbizu/forgottenserver/commits/version12_b

A regular rebase will make all the commits in your feature branch appear neatly organized after the commits in master. This is a pretty simple and straightforward.

So instead of this:

It becomes like this:

Then we can do interactive rebase to reorder, edit and squash various commits in your branch: https://www.youtube.com/watch?v=ElRzTuYln0M
To make your branch commit history tidy, clean and ready for merge.

If you have any questions using git, ask away, I'm happy to help.
Rebasing 74 commits is quite a big challenge.

lucasgrizante · 2021-11-09T22:17:00Z

Why this PR was closed?

I don't think #3784 should be merged as mostly work was done by Zbizu and not by Evil Puncker itself.

If you don't want to squash all your commits by topics, I think (but I'm no one in this community, so my opinion may not be the same as the moderators) you could at least make a single commit with a proper description (similar to the description from this PR).

You can do that doing git reset --soft origin/master and git commit again. Pretty easy (I'm sure you know it as you are an excellent programmer).

Znote · 2021-11-09T22:44:14Z

@lucasgrizante I agree with you. But I think Zbizu closed this PR because he don't want to continue it.

But perhaps he is working on a new branch to do the rebase or build a better commit structure. I did suggest to him to work on a version12_c branch. 🤞
(If this is the case, I also have a feeling its gonna be a juicy branch name). 🍺

Zbizu added 25 commits September 19, 2021 13:02

webService login support

7e4022b

logging in now possible both ways - legacy and webService\nClient 12 will crash on login because the server still sends 10.98 packets.

fix version 12 crash on receiving packet

3e85f0d

repaired 2FA + small adjustments

0dfc7cd

Ignore world name mismatch

e49cd19

Not necessary unless you run multiworld loginserver. Not present in current tfs either.

code cleanup

ed62e50

fixed disconnect message in older clients

305f15d

fix 10.98 disconnecting after first packet

6276cd9

small formatting fix

d79c5e5

replaced useless comment, removed unnecessary line

9f08062

restored previous connection constructor structure

6170f74

removing unused variable & correcting a typo

0122af9

better connection state names + code cleanup

3480058

experimental hybrid protocol

fc4be39

updated otb to version 12. You need special spr/dat to play with 10.98 now. Client 12 will still crash in most situations.

fix client icons for new protocol

1b89eed

bandaid fix for displaying monsters

96a60aa

protcol 12: in-game messages

aa0fdb6

addItem backwards compability

00e1266

new otb (credits: EPuncker)

1a7a18b

added version 12 bytes information

401392e

stabilizing protocol 12

af7960a

removed prints, disabled summon icon for protocol 12

a7059e8

fixed setOutfit box for new protocol

ab318b7

market no longer crashes client 12

869347a

fixed summon emblems (hopefully)

35a7ee6

code cleanup 1

44a8796

ghost added the feature New feature or functionality label Oct 18, 2021

ghost assigned ranisalt Oct 18, 2021

Farewell, legacy protocol!

65ba758

Zbizu added 4 commits October 26, 2021 12:42

show GUID on look

47153bd

update action bars on changes in player items

35b52ec

fix broken containers after playing from another location

b03f3a7

style consistency

e6c65f7

omarcopires mentioned this pull request Oct 26, 2021

[12.00] Adding support for colored loot #3761

Closed

3 tasks

TALKTYPE_SPELL and better compats

a09c4b7

nekiro reviewed Oct 27, 2021

View reviewed changes

src/const.h Outdated Show resolved Hide resolved

src/iologindata.cpp Show resolved Hide resolved

src/item.cpp Outdated Show resolved Hide resolved

src/item.cpp Outdated Show resolved Hide resolved

removing unnecessary casts

bcbe099

nekiro reviewed Nov 3, 2021

View reviewed changes

src/server.cpp Outdated Show resolved Hide resolved

removing unnecessary tautology

55b5d57

nekiro reviewed Nov 3, 2021

View reviewed changes

Zbizu added 2 commits November 3, 2021 18:10

code convention

b77b827

code cleanup 12

958346e

omarcopires mentioned this pull request Nov 4, 2021

[12.00] Adding support for colored loot #3771

Merged

3 tasks

added ability to read new otb flags (no official otb yet)

a37a310

ranisalt reviewed Nov 4, 2021

View reviewed changes

src/itemloader.h Outdated Show resolved Hide resolved

OTB 3.59 (credits: kokokoko)

aeeff53

ghost previously approved these changes Nov 4, 2021

View reviewed changes

removed deprecated feature

1c9424d

Zbizu dismissed ghost ’s stale review via 1c9424d November 6, 2021 15:13

ghost approved these changes Nov 6, 2021

View reviewed changes

Znote reviewed Nov 7, 2021

View reviewed changes

Znote requested changes Nov 7, 2021

View reviewed changes

ghost mentioned this pull request Nov 8, 2021

Version 12.71 update (second attempt) #3784

Closed

Zbizu closed this Nov 9, 2021

	DBResult_ptr result = db.storeQuery(fmt::format("SELECT `id`, `name`, `password`, `secret`, `type`, `premium_ends_at` FROM `accounts` WHERE `name` = {:s} OR `email` = {:s}", db.escapeString(name), db.escapeString(name)));
	DBResult_ptr result = db.storeQuery(fmt::format("SELECT `id`, `name`, `password`, `secret`, `type`, `premium_ends_at` FROM `accounts` WHERE {:s} IN(`name`, `email`)", db.escapeString(name)));

Version 12.71 update #3748

Version 12.71 update #3748

Uh oh!

Conversation

Zbizu commented Oct 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes Proposed

Credits

Uh oh!

Zbizu commented Oct 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mattyx14 commented Oct 19, 2021

Uh oh!

beats-dh commented Oct 26, 2021

Uh oh!

Zbizu commented Oct 26, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nekiro Nov 3, 2021

Choose a reason for hiding this comment

Uh oh!

Zbizu Nov 3, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nekiro Nov 3, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ranisalt Nov 5, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

Zbizu commented Nov 6, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Znote Nov 7, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ranisalt Nov 7, 2021

Choose a reason for hiding this comment

Uh oh!

Znote Nov 7, 2021

Choose a reason for hiding this comment

Uh oh!

Znote Nov 7, 2021

Choose a reason for hiding this comment

Uh oh!

nekiro Nov 7, 2021

Choose a reason for hiding this comment

Uh oh!

Znote left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lucasgrizante commented Nov 9, 2021

Uh oh!

Znote commented Nov 9, 2021

Uh oh!

Uh oh!

Zbizu commented Oct 18, 2021 •

edited

Loading

Zbizu commented Oct 19, 2021 •

edited

Loading

Zbizu commented Oct 26, 2021 •

edited

Loading

Zbizu Nov 3, 2021 •

edited

Loading

nekiro Nov 3, 2021 •

edited

Loading

Zbizu commented Nov 6, 2021 •

edited

Loading

Znote Nov 7, 2021 •

edited

Loading

Znote left a comment •

edited

Loading